!pip install keras==2.12.0

Collecting keras==2.12.0
  Downloading keras-2.12.0-py2.py3-none-any.whl (1.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 7.8 MB/s eta 0:00:00
Installing collected packages: keras
  Attempting uninstall: keras
    Found existing installation: keras 2.15.0
    Uninstalling keras-2.15.0:
      Successfully uninstalled keras-2.15.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow 2.15.0 requires keras<2.16,>=2.15.0, but you have keras 2.12.0 which is incompatible.
Successfully installed keras-2.12.0

!pip install tensorflow==2.10.0

Collecting tensorflow==2.10.0
  Downloading tensorflow-2.10.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (578.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 578.0/578.0 MB 2.7 MB/s eta 0:00:00
Requirement already satisfied: absl-py>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.10.0) (1.4.0)
Requirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.10.0) (1.6.3)
Requirement already satisfied: flatbuffers>=2.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.10.0) (24.3.25)
Collecting gast<=0.4.0,>=0.2.1 (from tensorflow==2.10.0)
  Downloading gast-0.4.0-py3-none-any.whl (9.8 kB)
Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.10.0) (0.2.0)
Requirement already satisfied: grpcio<2.0,>=1.24.3 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.10.0) (1.62.1)
Requirement already satisfied: h5py>=2.9.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.10.0) (3.9.0)
Collecting keras<2.11,>=2.10.0 (from tensorflow==2.10.0)
  Downloading keras-2.10.0-py2.py3-none-any.whl (1.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 92.5 MB/s eta 0:00:00
Collecting keras-preprocessing>=1.1.1 (from tensorflow==2.10.0)
  Downloading Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.6/42.6 kB 6.9 MB/s eta 0:00:00
Requirement already satisfied: libclang>=13.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.10.0) (18.1.1)
Requirement already satisfied: numpy>=1.20 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.10.0) (1.25.2)
Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.10.0) (3.3.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.10.0) (24.0)
Collecting protobuf<3.20,>=3.9.2 (from tensorflow==2.10.0)
  Downloading protobuf-3.19.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 77.6 MB/s eta 0:00:00
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.10.0) (67.7.2)
Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.10.0) (1.16.0)
Collecting tensorboard<2.11,>=2.10 (from tensorflow==2.10.0)
  Downloading tensorboard-2.10.1-py3-none-any.whl (5.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.9/5.9 MB 93.8 MB/s eta 0:00:00
Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.10.0) (0.36.0)
Collecting tensorflow-estimator<2.11,>=2.10.0 (from tensorflow==2.10.0)
  Downloading tensorflow_estimator-2.10.0-py2.py3-none-any.whl (438 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 438.7/438.7 kB 50.4 MB/s eta 0:00:00
Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.10.0) (2.4.0)
Requirement already satisfied: typing-extensions>=3.6.6 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.10.0) (4.10.0)
Requirement already satisfied: wrapt>=1.11.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.10.0) (1.14.1)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from astunparse>=1.6.0->tensorflow==2.10.0) (0.43.0)
Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.11,>=2.10->tensorflow==2.10.0) (2.27.0)
Collecting google-auth-oauthlib<0.5,>=0.4.1 (from tensorboard<2.11,>=2.10->tensorflow==2.10.0)
  Downloading google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.11,>=2.10->tensorflow==2.10.0) (3.6)
Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.11,>=2.10->tensorflow==2.10.0) (2.31.0)
Collecting tensorboard-data-server<0.7.0,>=0.6.0 (from tensorboard<2.11,>=2.10->tensorflow==2.10.0)
  Downloading tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl (4.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.9/4.9 MB 95.1 MB/s eta 0:00:00
Collecting tensorboard-plugin-wit>=1.6.0 (from tensorboard<2.11,>=2.10->tensorflow==2.10.0)
  Downloading tensorboard_plugin_wit-1.8.1-py3-none-any.whl (781 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 781.3/781.3 kB 70.3 MB/s eta 0:00:00
Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.11,>=2.10->tensorflow==2.10.0) (3.0.1)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.11,>=2.10->tensorflow==2.10.0) (5.3.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.11,>=2.10->tensorflow==2.10.0) (0.4.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.11,>=2.10->tensorflow==2.10.0) (4.9)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.11,>=2.10->tensorflow==2.10.0) (1.4.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.11,>=2.10->tensorflow==2.10.0) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.11,>=2.10->tensorflow==2.10.0) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.11,>=2.10->tensorflow==2.10.0) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.11,>=2.10->tensorflow==2.10.0) (2024.2.2)
Requirement already satisfied: MarkupSafe>=2.1.1 in /usr/local/lib/python3.10/dist-packages (from werkzeug>=1.0.1->tensorboard<2.11,>=2.10->tensorflow==2.10.0) (2.1.5)
Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.11,>=2.10->tensorflow==2.10.0) (0.6.0)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.11,>=2.10->tensorflow==2.10.0) (3.2.2)
Installing collected packages: tensorboard-plugin-wit, keras, tensorflow-estimator, tensorboard-data-server, protobuf, keras-preprocessing, gast, google-auth-oauthlib, tensorboard, tensorflow
  Attempting uninstall: keras
    Found existing installation: keras 2.15.0
    Uninstalling keras-2.15.0:
      Successfully uninstalled keras-2.15.0
  Attempting uninstall: tensorflow-estimator
    Found existing installation: tensorflow-estimator 2.15.0
    Uninstalling tensorflow-estimator-2.15.0:
      Successfully uninstalled tensorflow-estimator-2.15.0
  Attempting uninstall: tensorboard-data-server
    Found existing installation: tensorboard-data-server 0.7.2
    Uninstalling tensorboard-data-server-0.7.2:
      Successfully uninstalled tensorboard-data-server-0.7.2
  Attempting uninstall: protobuf
    Found existing installation: protobuf 3.20.3
    Uninstalling protobuf-3.20.3:
      Successfully uninstalled protobuf-3.20.3
  Attempting uninstall: gast
    Found existing installation: gast 0.5.4
    Uninstalling gast-0.5.4:
      Successfully uninstalled gast-0.5.4
  Attempting uninstall: google-auth-oauthlib
    Found existing installation: google-auth-oauthlib 1.2.0
    Uninstalling google-auth-oauthlib-1.2.0:
      Successfully uninstalled google-auth-oauthlib-1.2.0
  Attempting uninstall: tensorboard
    Found existing installation: tensorboard 2.15.2
    Uninstalling tensorboard-2.15.2:
      Successfully uninstalled tensorboard-2.15.2
  Attempting uninstall: tensorflow
    Found existing installation: tensorflow 2.15.0
    Uninstalling tensorflow-2.15.0:
      Successfully uninstalled tensorflow-2.15.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
pandas-gbq 0.19.2 requires google-auth-oauthlib>=0.7.0, but you have google-auth-oauthlib 0.4.6 which is incompatible.
tensorflow-datasets 4.9.4 requires protobuf>=3.20, but you have protobuf 3.19.6 which is incompatible.
tensorflow-metadata 1.14.0 requires protobuf<4.21,>=3.20.3, but you have protobuf 3.19.6 which is incompatible.
tf-keras 2.15.1 requires tensorflow<2.16,>=2.15, but you have tensorflow 2.10.0 which is incompatible.
Successfully installed gast-0.4.0 google-auth-oauthlib-0.4.6 keras-2.10.0 keras-preprocessing-1.1.2 protobuf-3.19.6 tensorboard-2.10.1 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.1 tensorflow-2.10.0 tensorflow-estimator-2.10.0

!pip install keras-tuner --upgrade

Collecting keras-tuner
  Downloading keras_tuner-1.4.7-py3-none-any.whl (129 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 129.1/129.1 kB 2.9 MB/s eta 0:00:00
Requirement already satisfied: keras in /usr/local/lib/python3.10/dist-packages (from keras-tuner) (2.10.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from keras-tuner) (24.0)
Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from keras-tuner) (2.31.0)
Collecting kt-legacy (from keras-tuner)
  Downloading kt_legacy-1.0.5-py3-none-any.whl (9.6 kB)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->keras-tuner) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->keras-tuner) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->keras-tuner) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->keras-tuner) (2024.2.2)
Installing collected packages: kt-legacy, keras-tuner
Successfully installed keras-tuner-1.4.7 kt-legacy-1.0.5

!pip install tensorflow scikeras scikit-learn

Requirement already satisfied: tensorflow in /usr/local/lib/python3.10/dist-packages (2.10.0)
Collecting scikeras
  Downloading scikeras-0.12.0-py3-none-any.whl (27 kB)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.10/dist-packages (1.2.2)
Requirement already satisfied: absl-py>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (1.4.0)
Requirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (1.6.3)
Requirement already satisfied: flatbuffers>=2.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (24.3.25)
Requirement already satisfied: gast<=0.4.0,>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (0.4.0)
Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (0.2.0)
Requirement already satisfied: grpcio<2.0,>=1.24.3 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (1.62.1)
Requirement already satisfied: h5py>=2.9.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (3.9.0)
Requirement already satisfied: keras<2.11,>=2.10.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (2.10.0)
Requirement already satisfied: keras-preprocessing>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (1.1.2)
Requirement already satisfied: libclang>=13.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (18.1.1)
Requirement already satisfied: numpy>=1.20 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (1.25.2)
Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (3.3.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from tensorflow) (24.0)
Requirement already satisfied: protobuf<3.20,>=3.9.2 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (3.19.6)
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from tensorflow) (67.7.2)
Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (1.16.0)
Requirement already satisfied: tensorboard<2.11,>=2.10 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (2.10.1)
Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (0.36.0)
Requirement already satisfied: tensorflow-estimator<2.11,>=2.10.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (2.10.0)
Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (2.4.0)
Requirement already satisfied: typing-extensions>=3.6.6 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (4.10.0)
Requirement already satisfied: wrapt>=1.11.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (1.14.1)
Requirement already satisfied: scipy>=1.3.2 in /usr/local/lib/python3.10/dist-packages (from scikit-learn) (1.11.4)
Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from scikit-learn) (1.3.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn) (3.4.0)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from astunparse>=1.6.0->tensorflow) (0.43.0)
Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.11,>=2.10->tensorflow) (2.27.0)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.11,>=2.10->tensorflow) (0.4.6)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.11,>=2.10->tensorflow) (3.6)
Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.11,>=2.10->tensorflow) (2.31.0)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.11,>=2.10->tensorflow) (0.6.1)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.11,>=2.10->tensorflow) (1.8.1)
Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.11,>=2.10->tensorflow) (3.0.1)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.11,>=2.10->tensorflow) (5.3.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.11,>=2.10->tensorflow) (0.4.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.11,>=2.10->tensorflow) (4.9)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.11,>=2.10->tensorflow) (1.4.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.11,>=2.10->tensorflow) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.11,>=2.10->tensorflow) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.11,>=2.10->tensorflow) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.11,>=2.10->tensorflow) (2024.2.2)
Requirement already satisfied: MarkupSafe>=2.1.1 in /usr/local/lib/python3.10/dist-packages (from werkzeug>=1.0.1->tensorboard<2.11,>=2.10->tensorflow) (2.1.5)
Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.11,>=2.10->tensorflow) (0.6.0)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.11,>=2.10->tensorflow) (3.2.2)
Installing collected packages: scikeras
Successfully installed scikeras-0.12.0

# Warnings
import warnings

# Numpy
import numpy as np

# Pandas
import pandas as pd

# Install unidecode and autocorrect
!pip install unidecode

# Autocorrect
!pip install autocorrect

# Contracted word expansion
!pip install contractions

# Visualisations
import matplotlib.pyplot as plt
import seaborn as sns
from PIL import Image
%matplotlib inline
sns.set_style('whitegrid')
sns.set_palette('viridis')

# Regex
import re

# String - to remove punctuations
import string

# Unidecode - removing accents and html tags
import unidecode

# Spell check
from autocorrect import Speller

# Contractions
import contractions

# NLTK
import nltk

# NLTK module downloads
nltk.download('punkt')
nltk.download('omw-1.4')
nltk.download('wordnet')
nltk.download('stopwords')
nltk.download('all')

# NLTK imports

# Stopwords
from nltk.corpus import stopwords

# Stemming
from nltk.stem.porter import PorterStemmer
from nltk.stem import LancasterStemmer
from nltk.stem import SnowballStemmer

# Lemmatization
from nltk.stem import WordNetLemmatizer

# Tokenization
from nltk.tokenize import word_tokenize, sent_tokenize

# spaCy
import spacy

# Instantiate the english web class
nlp = spacy.load('en_core_web_sm')

# spaCy tokenization
from spacy.lang.en import English
en_nlp = English()

## MACHINE LEARNING

# Train test split
from sklearn.model_selection import train_test_split

# Cross Validation, Hyperparameter Turning & Sampling
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import RandomizedSearchCV
from sklearn.model_selection import KFold
from sklearn.model_selection import StratifiedKFold
from sklearn.decomposition import PCA
from imblearn.over_sampling import SMOTE

# ML Models
import xgboost as xgb
from sklearn.ensemble import RandomForestClassifier
from sklearn.naive_bayes import MultinomialNB
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.svm import SVC
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.ensemble import AdaBoostClassifier
from sklearn.neighbors import KNeighborsClassifier

# Scalers & Encoders
from sklearn.preprocessing import MinMaxScaler
from sklearn.preprocessing import StandardScaler
from sklearn.preprocessing import LabelEncoder

# Sklearn Metrics
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix, precision_score, recall_score, f1_score, make_scorer, roc_auc_score, roc_curve

# Wordcloud
from wordcloud import WordCloud, STOPWORDS, ImageColorGenerator

# Installing additional modules/ libraries
from collections import Counter

# Vectorization Classes
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer

# Keras
import tensorflow as tf
from keras.datasets import imdb
from keras.models import Sequential
from tensorflow.keras.layers import Dense, Embedding, LSTM, SpatialDropout1D, Flatten, Dropout, Bidirectional, GlobalMaxPool1D
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint, ReduceLROnPlateau
from tensorflow.keras.initializers import Constant
from tensorflow.keras.models import load_model
from tensorflow.keras import optimizers
from keras.utils import to_categorical

# Word2Vec
from gensim.models import Word2Vec


# Keras Tuner for tuning
from kerastuner.tuners import RandomSearch

# Keras Classifier for tuning
from keras.wrappers.scikit_learn import KerasClassifier

Collecting unidecode
  Downloading Unidecode-1.3.8-py3-none-any.whl (235 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 235.5/235.5 kB 6.1 MB/s eta 0:00:00
Installing collected packages: unidecode
Successfully installed unidecode-1.3.8
Collecting autocorrect
  Downloading autocorrect-2.6.1.tar.gz (622 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 622.8/622.8 kB 9.4 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: autocorrect
  Building wheel for autocorrect (setup.py) ... done
  Created wheel for autocorrect: filename=autocorrect-2.6.1-py3-none-any.whl size=622363 sha256=0109310e5fd186ccf3c482cb059611d2a976298cf8743d53d240bd98dd3b32c0
  Stored in directory: /root/.cache/pip/wheels/b5/7b/6d/b76b29ce11ff8e2521c8c7dd0e5bfee4fb1789d76193124343
Successfully built autocorrect
Installing collected packages: autocorrect
Successfully installed autocorrect-2.6.1
Collecting contractions
  Downloading contractions-0.1.73-py2.py3-none-any.whl (8.7 kB)
Collecting textsearch>=0.0.21 (from contractions)
  Downloading textsearch-0.0.24-py2.py3-none-any.whl (7.6 kB)
Collecting anyascii (from textsearch>=0.0.21->contractions)
  Downloading anyascii-0.3.2-py3-none-any.whl (289 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 289.9/289.9 kB 4.7 MB/s eta 0:00:00
Collecting pyahocorasick (from textsearch>=0.0.21->contractions)
  Downloading pyahocorasick-2.1.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (110 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 110.7/110.7 kB 6.8 MB/s eta 0:00:00
Installing collected packages: pyahocorasick, anyascii, textsearch, contractions
Successfully installed anyascii-0.3.2 contractions-0.1.73 pyahocorasick-2.1.0 textsearch-0.0.24

[nltk_data] Downloading package punkt to /root/nltk_data...
[nltk_data]   Unzipping tokenizers/punkt.zip.
[nltk_data] Downloading package omw-1.4 to /root/nltk_data...
[nltk_data] Downloading package wordnet to /root/nltk_data...
[nltk_data] Downloading package stopwords to /root/nltk_data...
[nltk_data]   Unzipping corpora/stopwords.zip.
[nltk_data] Downloading collection 'all'
[nltk_data]    | 
[nltk_data]    | Downloading package abc to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/abc.zip.
[nltk_data]    | Downloading package alpino to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/alpino.zip.
[nltk_data]    | Downloading package averaged_perceptron_tagger to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping taggers/averaged_perceptron_tagger.zip.
[nltk_data]    | Downloading package averaged_perceptron_tagger_ru to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping
[nltk_data]    |       taggers/averaged_perceptron_tagger_ru.zip.
[nltk_data]    | Downloading package basque_grammars to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping grammars/basque_grammars.zip.
[nltk_data]    | Downloading package bcp47 to /root/nltk_data...
[nltk_data]    | Downloading package biocreative_ppi to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/biocreative_ppi.zip.
[nltk_data]    | Downloading package bllip_wsj_no_aux to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping models/bllip_wsj_no_aux.zip.
[nltk_data]    | Downloading package book_grammars to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping grammars/book_grammars.zip.
[nltk_data]    | Downloading package brown to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/brown.zip.
[nltk_data]    | Downloading package brown_tei to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/brown_tei.zip.
[nltk_data]    | Downloading package cess_cat to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/cess_cat.zip.
[nltk_data]    | Downloading package cess_esp to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/cess_esp.zip.
[nltk_data]    | Downloading package chat80 to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/chat80.zip.
[nltk_data]    | Downloading package city_database to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/city_database.zip.
[nltk_data]    | Downloading package cmudict to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/cmudict.zip.
[nltk_data]    | Downloading package comparative_sentences to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/comparative_sentences.zip.
[nltk_data]    | Downloading package comtrans to /root/nltk_data...
[nltk_data]    | Downloading package conll2000 to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/conll2000.zip.
[nltk_data]    | Downloading package conll2002 to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/conll2002.zip.
[nltk_data]    | Downloading package conll2007 to /root/nltk_data...
[nltk_data]    | Downloading package crubadan to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/crubadan.zip.
[nltk_data]    | Downloading package dependency_treebank to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/dependency_treebank.zip.
[nltk_data]    | Downloading package dolch to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/dolch.zip.
[nltk_data]    | Downloading package europarl_raw to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/europarl_raw.zip.
[nltk_data]    | Downloading package extended_omw to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    | Downloading package floresta to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/floresta.zip.
[nltk_data]    | Downloading package framenet_v15 to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/framenet_v15.zip.
[nltk_data]    | Downloading package framenet_v17 to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/framenet_v17.zip.
[nltk_data]    | Downloading package gazetteers to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/gazetteers.zip.
[nltk_data]    | Downloading package genesis to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/genesis.zip.
[nltk_data]    | Downloading package gutenberg to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/gutenberg.zip.
[nltk_data]    | Downloading package ieer to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/ieer.zip.
[nltk_data]    | Downloading package inaugural to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/inaugural.zip.
[nltk_data]    | Downloading package indian to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/indian.zip.
[nltk_data]    | Downloading package jeita to /root/nltk_data...
[nltk_data]    | Downloading package kimmo to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/kimmo.zip.
[nltk_data]    | Downloading package knbc to /root/nltk_data...
[nltk_data]    | Downloading package large_grammars to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping grammars/large_grammars.zip.
[nltk_data]    | Downloading package lin_thesaurus to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/lin_thesaurus.zip.
[nltk_data]    | Downloading package mac_morpho to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/mac_morpho.zip.
[nltk_data]    | Downloading package machado to /root/nltk_data...
[nltk_data]    | Downloading package masc_tagged to /root/nltk_data...
[nltk_data]    | Downloading package maxent_ne_chunker to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping chunkers/maxent_ne_chunker.zip.
[nltk_data]    | Downloading package maxent_treebank_pos_tagger to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping taggers/maxent_treebank_pos_tagger.zip.
[nltk_data]    | Downloading package moses_sample to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping models/moses_sample.zip.
[nltk_data]    | Downloading package movie_reviews to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/movie_reviews.zip.
[nltk_data]    | Downloading package mte_teip5 to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/mte_teip5.zip.
[nltk_data]    | Downloading package mwa_ppdb to /root/nltk_data...
[nltk_data]    |   Unzipping misc/mwa_ppdb.zip.
[nltk_data]    | Downloading package names to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/names.zip.
[nltk_data]    | Downloading package nombank.1.0 to /root/nltk_data...
[nltk_data]    | Downloading package nonbreaking_prefixes to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/nonbreaking_prefixes.zip.
[nltk_data]    | Downloading package nps_chat to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/nps_chat.zip.
[nltk_data]    | Downloading package omw to /root/nltk_data...
[nltk_data]    | Downloading package omw-1.4 to /root/nltk_data...
[nltk_data]    |   Package omw-1.4 is already up-to-date!
[nltk_data]    | Downloading package opinion_lexicon to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/opinion_lexicon.zip.
[nltk_data]    | Downloading package panlex_swadesh to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    | Downloading package paradigms to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/paradigms.zip.
[nltk_data]    | Downloading package pe08 to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/pe08.zip.
[nltk_data]    | Downloading package perluniprops to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping misc/perluniprops.zip.
[nltk_data]    | Downloading package pil to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/pil.zip.
[nltk_data]    | Downloading package pl196x to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/pl196x.zip.
[nltk_data]    | Downloading package porter_test to /root/nltk_data...
[nltk_data]    |   Unzipping stemmers/porter_test.zip.
[nltk_data]    | Downloading package ppattach to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/ppattach.zip.
[nltk_data]    | Downloading package problem_reports to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/problem_reports.zip.
[nltk_data]    | Downloading package product_reviews_1 to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/product_reviews_1.zip.
[nltk_data]    | Downloading package product_reviews_2 to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/product_reviews_2.zip.
[nltk_data]    | Downloading package propbank to /root/nltk_data...
[nltk_data]    | Downloading package pros_cons to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/pros_cons.zip.
[nltk_data]    | Downloading package ptb to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/ptb.zip.
[nltk_data]    | Downloading package punkt to /root/nltk_data...
[nltk_data]    |   Package punkt is already up-to-date!
[nltk_data]    | Downloading package qc to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/qc.zip.
[nltk_data]    | Downloading package reuters to /root/nltk_data...
[nltk_data]    | Downloading package rslp to /root/nltk_data...
[nltk_data]    |   Unzipping stemmers/rslp.zip.
[nltk_data]    | Downloading package rte to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/rte.zip.
[nltk_data]    | Downloading package sample_grammars to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping grammars/sample_grammars.zip.
[nltk_data]    | Downloading package semcor to /root/nltk_data...
[nltk_data]    | Downloading package senseval to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/senseval.zip.
[nltk_data]    | Downloading package sentence_polarity to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/sentence_polarity.zip.
[nltk_data]    | Downloading package sentiwordnet to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/sentiwordnet.zip.
[nltk_data]    | Downloading package shakespeare to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/shakespeare.zip.
[nltk_data]    | Downloading package sinica_treebank to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/sinica_treebank.zip.
[nltk_data]    | Downloading package smultron to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/smultron.zip.
[nltk_data]    | Downloading package snowball_data to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    | Downloading package spanish_grammars to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping grammars/spanish_grammars.zip.
[nltk_data]    | Downloading package state_union to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/state_union.zip.
[nltk_data]    | Downloading package stopwords to /root/nltk_data...
[nltk_data]    |   Package stopwords is already up-to-date!
[nltk_data]    | Downloading package subjectivity to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/subjectivity.zip.
[nltk_data]    | Downloading package swadesh to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/swadesh.zip.
[nltk_data]    | Downloading package switchboard to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/switchboard.zip.
[nltk_data]    | Downloading package tagsets to /root/nltk_data...
[nltk_data]    |   Unzipping help/tagsets.zip.
[nltk_data]    | Downloading package timit to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/timit.zip.
[nltk_data]    | Downloading package toolbox to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/toolbox.zip.
[nltk_data]    | Downloading package treebank to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/treebank.zip.
[nltk_data]    | Downloading package twitter_samples to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/twitter_samples.zip.
[nltk_data]    | Downloading package udhr to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/udhr.zip.
[nltk_data]    | Downloading package udhr2 to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/udhr2.zip.
[nltk_data]    | Downloading package unicode_samples to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping corpora/unicode_samples.zip.
[nltk_data]    | Downloading package universal_tagset to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping taggers/universal_tagset.zip.
[nltk_data]    | Downloading package universal_treebanks_v20 to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    | Downloading package vader_lexicon to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    | Downloading package verbnet to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/verbnet.zip.
[nltk_data]    | Downloading package verbnet3 to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/verbnet3.zip.
[nltk_data]    | Downloading package webtext to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/webtext.zip.
[nltk_data]    | Downloading package wmt15_eval to /root/nltk_data...
[nltk_data]    |   Unzipping models/wmt15_eval.zip.
[nltk_data]    | Downloading package word2vec_sample to
[nltk_data]    |     /root/nltk_data...
[nltk_data]    |   Unzipping models/word2vec_sample.zip.
[nltk_data]    | Downloading package wordnet to /root/nltk_data...
[nltk_data]    |   Package wordnet is already up-to-date!
[nltk_data]    | Downloading package wordnet2021 to /root/nltk_data...
[nltk_data]    | Downloading package wordnet2022 to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/wordnet2022.zip.
[nltk_data]    | Downloading package wordnet31 to /root/nltk_data...
[nltk_data]    | Downloading package wordnet_ic to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/wordnet_ic.zip.
[nltk_data]    | Downloading package words to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/words.zip.
[nltk_data]    | Downloading package ycoe to /root/nltk_data...
[nltk_data]    |   Unzipping corpora/ycoe.zip.
[nltk_data]    | 
[nltk_data]  Done downloading collection all
<ipython-input-3-435cdfbeebf1>:139: DeprecationWarning: `import kerastuner` is deprecated, please use `import keras_tuner`.
  from kerastuner.tuners import RandomSearch

Configurations

# Pandas column width
pd.set_option('max_colwidth', None)

# Seaborn palette
sns.set_style('whitegrid')
sns.set_palette('viridis')

# Random number seed
seed = 343
maxlen=200
max_features=1000

# Warnings
warnings.filterwarnings('ignore')

Functions

# Function to see the unique values of a column, and to plot frequency graphs

def univariate_analysis(dataframe, column, normalize_data = False):

    '''
    Inputs:
    dataframe --> array
    column --> str: name of column to be analysed
    normalize_data --> If True, prints converts value counts of data into percentages. Default = False

    Output:
    1. Count of unique values in the column
    2. Unique values in the column
    3. Value counts of unique values in the column
    4. Histogram of data if data is continuous
    5. Barchart of data if data is categorical or discrete
    '''

    # Getting the number of unique values
    print(f"The column '{column}' has {dataframe[column].nunique()} unique values.\n")
    print(f"Unique values are:\n")
    print(f"{dataframe[column].unique()}\n")
    print(f"The value counts of data in this column are:\n{dataframe[column].value_counts(normalize = normalize_data)}")

    # Plotting graphs
    if dataframe[column].dtype == 'object':
        fig, (ax1, ax2) = plt.subplots(nrows = 1, ncols = 2, figsize = (20,6))
        ax1.set_title(f"Countplot of data in column {column}.", fontsize = 12, pad = 15)
        sns.countplot(data = dataframe, x = column, ax = ax1)
        ax1.set_xlabel(f"Column: {column}", fontsize = 12, labelpad = 12)
        ax1.set_ylabel(f"Count", fontsize = 12, labelpad = 12)
        if len(dataframe[column].value_counts()) > 3:
            ax1.set_xticklabels(ax1.get_xticklabels(), rotation = 90)
        ax2.set_title(f"Pie Chart: {column}.", fontsize = 12, pad = 15)
        ax2.pie(dataframe[column].value_counts(),
                autopct = "%.1f",
                labels = dataframe[column].value_counts().index,
                shadow = True,
                explode = [0.1]*len(dataframe[column].value_counts().index),
                startangle = -135)
        plt.show()
    elif dataframe[column].dtype == 'int' or dataframe[column].dtype == 'float':
        fig, (ax1, ax2) = plt.subplots(nrows = 1, ncols = 2, figsize = (20,6))
        ax1.set_title(f"Histogram of data in column {column}.", fontsize = 12, pad = 15)
        sns.histplot(data = dataframe, x = column, kde = True, ax = ax1)
        ax1.set_xlabel(f"Column: {column}", fontsize = 12, labelpad = 12)
        ax1.set_ylabel(f"Frequency", fontsize = 12, labelpad = 12)
        ax2.set_title(f"Pie Chart: {column}.", fontsize = 12, pad = 15)
        ax2.pie(dataframe[column].value_counts(),
                autopct = "%.1f",
                labels = dataframe[column].value_counts().index,
                shadow = True,
                explode = [0.1]*len(dataframe[column].value_counts().index),
                startangle = -135)
        plt.show()

# Function to create bivariate plots

def bivariate_analysis(dataframe, variable_1, variable_2):

    '''
    Inputs:
    dataframe --> array
    variable_1 --> str: name of column to be plotted on the x-axis
    variable_2 --> str: name of column by which variable_1 is split, corresponding to variable_1 's representation in variable_2

    Output:
    Barchart of variable_1 on the x axis, split by its value counts in variable_2
    '''

    fig, ax = plt.subplots(nrows = 1, ncols = 1, figsize = (30,6))
    ax.set_title(f"Bivariate Countplot: {variable_1} & {variable_2}", fontsize = 12, pad = 15)
    sns.countplot(data = dataframe, x = variable_1, hue = variable_2, ax = ax)
    ax.set_xlabel(f"Column: {variable_1}", fontsize = 12, labelpad = 12)
    ax.set_ylabel(f"Column: {variable_2}", fontsize = 12, labelpad = 12)
    if len(dataframe[variable_1].value_counts()) > 3:
        ax.set_xticklabels(ax.get_xticklabels(), rotation = 90)
    plt.show()

# Function to clean textual data

def clean_text(df, punctuations = '''!()-[]{};:'"\,<>./?@#$%^&*_~'''):

    '''
    Inputs:
    dataframe --> array

    Output:
    text: str, cleaned for
    1. URL & HTML tags
    2. Any element that is not a word or whitespace character
    3. Converting text to lowercase
    4. Expanding contracted words
    5. Removing stopwords, except for the word 'not','while', 'when', 'during', since this word would provide context to sentences
    6. Unncessary whitespaces around words
    7. numbers
    8. Lemmatizing the words
    9. Removing words with length = 1
    '''

    # Cleaning URLS
    string = re.sub(r'https?://\S+|www\.\S+', '', df)

    # Cleaning HTML elements
    string = re.sub(r'<.*?>', '', df)

    # Remove anything that is not a word or whitespace character
    string = re.sub(r'[^\w\s]', '', df)

    # Converting text to lowercase
    string = string.lower()

    # Fix contractions
    string = ' '.join([contractions.fix(word) for word in string.split(' ')])

    # Removing stop words, except for the word 'not'
    stopwords_list = set(nltk.corpus.stopwords.words('english'))
    stopwords_list.remove('not')
    stopwords_list.remove('while')
    stopwords_list.remove('when')
    stopwords_list.remove('during')
    # stopwords_list.remove('by')
    stopwords_list.remove('between')
    string = ' '.join([word for word in string.split() if word not in stopwords_list])

    # Cleaning whitespaces
    string = re.sub(r'\s+', ' ', string).strip()

    # Remove numbers
    string = ' '.join([word for word in string.split() if word.isalpha()])

    # Remove words of length 1
    string = [word for word in string.split() if len(word) > 1]

    # Lemmatize
    string = ' '.join([WordNetLemmatizer().lemmatize(word) for word in string])

    return string

# Function to get the length of sentences in a dataframe column

def sent_len(dataframe, col_to_assess):

    '''
    Inputs:
    dataframe --> Array
    col_to_assess: str, name of column with text whose length is to be determined

    Output:
    list: containing the total number of words in the specified column for each row of the specified dataframe
    '''

    sent_len_list = []

    for i in range(dataframe.shape[0]):

        text = dataframe[col_to_assess][i].strip()  # Remove leading and trailing whitespaces
        split_sent = text.split(' ')  # Use space as the delimiter
        split_sent_len = len(split_sent)
        sent_len_list.append(split_sent_len)

        # split_sent = dataframe[col_to_assess][i].split('')
        # split_sent_len = len(split_sent)
        # sent_len_list.append(split_sent_len)

    return sent_len_list

# Functions that will return the confusion matrix of a trained model

def get_confusion_matrix(model_name, y_training_actual, y_testing_actual, y_training_predictions, y_testing_predictions):

    '''
    '''

    model_name = str(model_name)

    conf_matrix_train = metrics.confusion_matrix(y_training_actual, y_training_predictions)
    conf_matrix_test = metrics.confusion_matrix(y_testing_actual, y_testing_predictions)

    df_conf_matrix_train = pd.DataFrame(conf_matrix_train, index = ['Actual Pass', 'Actual Fail'], columns = ['Predicted Pass', 'Predicted Fail'])
    df_conf_matrix_test = pd.DataFrame(conf_matrix_test, index = ['Actual Pass', 'Actual Fail'], columns = ['Predicted Pass', 'Predicted Fail'])

    conf_matrix_train_valuelabels = list(conf_matrix_train.ravel())
    conf_matrix_train_valuepercent = ["{0:.2%}".format(value) for value in list(conf_matrix_train.flatten())/np.sum(conf_matrix_train)]
    conf_matrix_train_labels = [f"{i}\n{j}" for i, j in zip(conf_matrix_train_valuelabels, conf_matrix_train_valuepercent)]
    conf_matrix_train_labels = np.asarray(conf_matrix_train_labels).reshape(2,2)

    conf_matrix_test_valuelabels = list(conf_matrix_test.ravel())
    conf_matrix_test_valuepercent = ["{0:.2%}".format(value) for value in list(conf_matrix_test.flatten())/np.sum(conf_matrix_test)]
    conf_matrix_test_labels = [f"{i}\n{j}" for i, j in zip(conf_matrix_test_valuelabels, conf_matrix_test_valuepercent)]
    conf_matrix_test_labels = np.asarray(conf_matrix_test_labels).reshape(2,2)

    # Plotting
    fig, (ax1, ax2) = plt.subplots(nrows = 1, ncols = 2, figsize = (30, 6))
    fig.suptitle(f"Confusion Matrix of {model_name} model: Train & Test Sets", fontsize = 20)

    sns.heatmap(data = df_conf_matrix_train, annot = conf_matrix_train_labels, fmt = '', ax = ax1)
    ax1.set_title("Training Set", fontsize = 15, pad = 10)
    ax1.set_xlabel("Predicted Labels", fontsize = 15, labelpad = 10)
    ax1.set_ylabel("Actual Labels", fontsize = 15, labelpad = 10)

    sns.heatmap(data = df_conf_matrix_test, annot = conf_matrix_test_labels, fmt = '', ax = ax2)
    ax2.set_title("Test Set", fontsize = 15, pad = 10)
    ax2.set_xlabel("Predicted Labels", fontsize = 15, labelpad = 10)
    ax2.set_ylabel("Actual Labels", fontsize = 15, labelpad = 10)

# Writing a function that will return the performance report of a trained model

def model_performance_eval(model_name, y_training_actual, y_testing_actual, y_training_predictions, y_testing_predictions):

    '''
    '''

    model_name = str(model_name)

    print(f"Performance Metrics:\n\nMODEL: {model_name}\n")

    print(f"Training Set Accuracy: {round((accuracy_score(y_training_actual, y_training_predictions)*100), 2)}%\n")
    print(f"Test Set Accuracy: {round((accuracy_score(y_testing_actual, y_testing_predictions)*100), 2)}%\n")

    print("Classification Report: Training Set\n", classification_report(y_training_actual, y_training_predictions), "\n")
    print("Classification Report: Test Set\n", classification_report(y_testing_actual, y_testing_predictions), "\n")

    train_perf_dict = classification_report(y_training_actual, y_training_predictions, output_dict = True)['1']
    train_perf_dict['Accuracy'] = classification_report(y_training_actual, y_training_predictions, output_dict = True)['accuracy']
    train_perf_df = pd.DataFrame(train_perf_dict, index = [model_name])

    test_perf_dict = classification_report(y_testing_actual, y_testing_predictions, output_dict = True)['1']
    test_perf_dict['Accuracy'] = classification_report(y_testing_actual, y_testing_predictions, output_dict = True)['accuracy']
    test_perf_df = pd.DataFrame(test_perf_dict, index = [model_name])

    return train_perf_df, test_perf_df

def crossvalscore_report(model, X_training, y_training, cvstrat, seed):

    '''
    '''

    kfold = KFold(n_splits = 10, shuffle = True, random_state = seed)
    stratkfold = StratifiedKFold(n_splits = 10, shuffle = True, random_state = seed)

    if cvstrat == 'KFold':
        cv = kfold
    elif cvstrat == 'Stratified':
        cv = stratkfold

    cv_acc = cross_val_score(estimator = model,
                             X = X_training,
                             y = y_training,
                             cv = cv,
                             scoring = 'accuracy',
                             n_jobs = -1)

    cv_rec = cross_val_score(estimator = model,
                             X = X_training,
                             y = y_training,
                             cv = cv,
                             scoring = 'recall',
                             n_jobs = -1)

    cv_pre = cross_val_score(estimator = model,
                             X = X_training,
                             y = y_training,
                             cv = cv,
                             scoring = 'precision',
                             n_jobs = -1)

    cv_f1 = cross_val_score(estimator = model,
                             X = X_training,
                             y = y_training,
                             cv = cv,
                             scoring = 'f1',
                             n_jobs = -1)

    print(f"Using {cvstrat}, the average cross validation accuracy score on the {model} model is {cv_acc.mean()}, with a standard deviation of {cv_acc.std()}\n")
    # print(f"Using {cvstrat}, the average cross validation recall score of the minority class in the {model} model is {cv_rec.mean()} with a standard deviation of {cv_rec.std()}\n")
    # print(f"Using {cvstrat}, the average cross validation precision score of the minority class in the {model} model is {cv_pre.mean()} with a standard deviation of {cv_pre.std()}\n")
    # print(f"Using {cvstrat}, the average cross validation f1 score of the minority class in the {model} model is {cv_f1.mean()} with a standard deviation of {cv_f1.std()}\n")

    accuracies = cv_acc
    # recall = cv_rec
    # precision = cv_pre
    # f1score = cv_f1

    avg_acc = cv_acc.mean()
    # avg_rec = cv_rec.mean()
    # avg_pre = cv_pre.mean()
    # avg_f1 = cv_f1.mean()

    std_acc = cv_acc.std()
    # std_rec = cv_rec.std()
    # std_pre = cv_pre.std()
    # std_f1 = cv_f1.std()

    acc_pair = [avg_acc, std_acc]
    # rec_pair = [avg_rec, std_rec]
    # pre_pair = [avg_pre, std_pre]
    # f1_pair = [avg_f1, std_f1]

    return acc_pair # rec_pair, pre_pair, f1_pair

##Create standard function to include all Machine learning models
##A dictionary will keep all accuracy for all the models and it can be used for comparative analysis
##This function will be called for types of train and test data

def ML_Models(X_train, X_test, y_train, y_test):

  models={
      "LogisticRegression":LogisticRegression(multi_class='multinomial',solver='saga', max_iter=10000,random_state = seed),
      "Multinomial NB": MultinomialNB(),
      "KNearestNeighbors": KNeighborsClassifier(),
      "DecisionTreeClassifier":DecisionTreeClassifier(criterion='entropy',class_weight = 'balanced', max_depth = 6,random_state = seed, min_samples_leaf = 5),
      "RandomForestClassifier":RandomForestClassifier(class_weight = 'balanced',n_estimators=100, max_depth = 5,random_state = seed),
      "AdaBoostClassifier":AdaBoostClassifier(random_state = seed),
      "GradientBoostClassifier":GradientBoostingClassifier(random_state = seed),
      'XGBoostClassifier': xgb.XGBClassifier(colsample_bytree= 0.6, gamma= 0.5, max_depth= 5, min_child_weight= 1, subsample= 0.8,n_estimators=100)
  }

  names = []
  train_scores = []
  test_scores = []

  for name, model in models.items():
    model.fit(X_train, y_train)
    result_train = accuracy_score(y_train, model.predict(X_train))
    result_test = accuracy_score(y_test, model.predict(X_test))
    names.append(name)
    train_scores.append(result_train) # Appending the test scores to the list
    test_scores.append(result_test) # Appending the test scores to the list
    result_df = pd.DataFrame({'model': names, 'Train accuracy': train_scores, 'Test accuracy': test_scores})

    cm = confusion_matrix(y_test, model.predict(X_test))

    fig, ax = plt.subplots(nrows = 1, ncols = 1, figsize = (5,4))
    sns.heatmap(cm,
                annot=True,
                fmt='g',
                xticklabels=['I','II','III','IV','V'],
                yticklabels=['I','II','III','IV','V'], ax = ax)
    ax.set_ylabel('Prediction', fontsize = 11)
    ax.set_xlabel('Actual',fontsize = 11)
    ax.set_title(f'Confusion Matrix: {name} model',fontsize = 14)
    plt.show()

  return result_df

##Create standard function to include all Machine learning models - without Multinomial NB since it doesn't take negative Word2Vec values
##A dictionary will keep all accuracy for all the models and it can be used for comparative analysis
##This function will be called for types of train and test data

def ML_Models_without_MNB(X_train, X_test, y_train, y_test):

  models={
      "LogisticRegression":LogisticRegression(multi_class='multinomial',solver='saga', max_iter=10000,random_state = seed),
    #   "Multinomial NB": MultinomialNB(),
      "KNearestNeighbors": KNeighborsClassifier(),
      "DecisionTreeClassifier":DecisionTreeClassifier(criterion='entropy',class_weight = 'balanced', max_depth = 6,random_state = seed, min_samples_leaf = 5),
      "RandomForestClassifier":RandomForestClassifier(class_weight = 'balanced',n_estimators=100, max_depth = 5,random_state = seed),
      "AdaBoostClassifier":AdaBoostClassifier(random_state = seed),
      "GradientBoostClassifier":GradientBoostingClassifier(random_state = seed),
      'XGBoostClassifier': xgb.XGBClassifier(colsample_bytree= 0.6, gamma= 0.5, max_depth= 5, min_child_weight= 1, subsample= 0.8,n_estimators=100)
  }

  names = []
  train_scores = []
  test_scores = []

  for name, model in models.items():
    model.fit(X_train, y_train)
    result_train = accuracy_score(y_train, model.predict(X_train))
    result_test = accuracy_score(y_test, model.predict(X_test))
    names.append(name)
    train_scores.append(result_train) # Appending the test scores to the list
    test_scores.append(result_test) # Appending the test scores to the list
    result_df = pd.DataFrame({'model': names, 'Train accuracy': train_scores, 'Test accuracy': test_scores})

    cm = confusion_matrix(y_test, model.predict(X_test))
    fig, ax = plt.subplots(nrows = 1, ncols = 1, figsize = (5,4))
    sns.heatmap(cm,
                annot=True,
                fmt='g',
                xticklabels=['I','II','III','IV','V'],
                yticklabels=['I','II','III','IV','V'], ax = ax)
    ax.set_ylabel('Prediction', fontsize = 11)
    ax.set_xlabel('Actual',fontsize = 11)
    ax.set_title(f'Confusion Matrix: {name} model',fontsize = 14)
    plt.show()

  return result_df

def gridsearchcv(model,param_grid,X_train,X_test,y_train,y_test):

  # num_folds = 10
  # kfold = KFold(n_splits=num_folds)

  stratified_kfold = StratifiedKFold(n_splits=10,
                                       shuffle=True,
                                       random_state = seed)

  clf = GridSearchCV(model, param_grid, cv=stratified_kfold,n_jobs=-1)
  clf.fit(X_train, y_train)
  test_score = clf.score(X_test, y_test)
  train_score = clf.score(X_train, y_train)

  return train_score,test_score,clf.best_params_

def ML_Tuned_Models(X_train,X_test,y_train,y_test):

  models={
      "RanForCls":RandomForestClassifier(),
      "AdaBoosCls":AdaBoostClassifier(),
    #   "GradBoosCls":GradientBoostingClassifier(),
    #   'XGBoost': xgb.XGBClassifier(),
      "LogReg":LogisticRegression(),
      "KNN": KNeighborsClassifier(),
      "Multinomial NB": MultinomialNB(),

  }
  AdaBoosCls_param_grid = {
     'n_estimators': [10, 50, 100, 500],
     'learning_rate': [0.01, 0.05, 0.1, 1],
  }

  GradBoosCls_param_grid = {
    "learning_rate": [0.01, 0.025, 0.05],
    "max_depth":[3,5,8],
    "n_estimators":[10, 100, 1000]
  }

  RanForCls_param_grid = {'max_features': ['sqrt', 'log2'],
              'ccp_alpha': [0.1, .01, .001],
              'max_depth' : [7, 8, 9, 10, 11, 12],
              'criterion' :['gini', 'entropy']
             }

  XGBoost_param_grid= {
        'n_estimators': [10, 50, 100, 500],
        # 'min_child_weight': [1, 5, 10],
        'gamma': [0.5, 1, 1.5, 2, 5],
        # 'subsample': [0.6, 0.8, 1.0],
        # 'colsample_bytree': [0.6, 0.8, 1.0],
        'max_depth': [3, 4, 5]
  }

  LogReg_param_grid = {
    'penalty': ['l1', 'l2'],
    'C': [0.1, 1, 10, 100]
  }

  KNN_param_grid = {
    'n_neighbors' : range(1, 21, 2),
    'weights' : ['uniform', 'distance'],
    'metric' : ['euclidean', 'manhattan', 'minkowski']
  }

  MNB_grid_params = {
  'alpha': np.linspace(0.5, 1.5, 6),
  'fit_prior': [True, False],
  }

  names = []
  train_scores = []
  test_scores = []

  for name, model in models.items():
    if name=='AdaBoosCls':
      param_grid=AdaBoosCls_param_grid
    elif name=='GradBoosCls':
      param_grid=GradBoosCls_param_grid
    elif name=='RanForCls':
      param_grid=RanForCls_param_grid
    elif name=='XGBoost':
      param_grid=XGBoost_param_grid
    elif name=='KNN':
      param_grid=KNN_param_grid
    elif name=='LogReg':
      param_grid=LogReg_param_grid
    elif name=='Multinomial NB':
      param_grid=MNB_grid_params

    train_score,test_score,best_parameters=gridsearchcv(model,param_grid,X_train,X_test,y_train,y_test)
    names.append(name)
    train_scores.append(train_score) # Appending the test scores to the list
    test_scores.append(test_score) # Appending the test scores to the list
    result_tuned_df= pd.DataFrame({'model': names, 'train best score': train_scores, 'test best score': test_scores })
    print(' model: '+ name + '\n best parameters: ' )
    print( best_parameters)

  return result_tuned_df

def ML_Tuned_Models_without_MNB(X_train,X_test,y_train,y_test):

  models={
      "RanForCls":RandomForestClassifier(),
      "AdaBoosCls":AdaBoostClassifier(),
    #   "GradBoosCls":GradientBoostingClassifier(),
    #   'XGBoost': XGBClassifier(),
      "LogReg":LogisticRegression(),
      "KNN": KNeighborsClassifier(),
    #   "Multinomial NB": MultinomialNB(),

  }
  AdaBoosCls_param_grid = {
     'n_estimators': [10, 50, 100, 500],
     'learning_rate': [0.01, 0.05, 0.1, 1],
  }

  GradBoosCls_param_grid = {
    "learning_rate": [0.01, 0.025, 0.05],
    "max_depth":[3,5,8],
    "n_estimators":[10, 100, 1000]
  }

  RanForCls_param_grid = {'max_features': ['sqrt', 'log2'],
              'ccp_alpha': [0.1, .01, .001],
              'max_depth' : [7, 8, 9, 10, 11, 12],
              'criterion' :['gini', 'entropy']
             }

  XGBoost_param_grid= {
        'n_estimators': [10, 50, 100, 500],
        # 'min_child_weight': [1, 5, 10],
        'gamma': [0.5, 1, 1.5, 2, 5],
        # 'subsample': [0.6, 0.8, 1.0],
        # 'colsample_bytree': [0.6, 0.8, 1.0],
        'max_depth': [3, 4, 5]
  }

  LogReg_param_grid = {
    'penalty': ['l1', 'l2'],
    'C': [0.1, 1, 10, 100]
  }

  KNN_param_grid = {
    'n_neighbors' : range(1, 21, 2),
    'weights' : ['uniform', 'distance'],
    'metric' : ['euclidean', 'manhattan', 'minkowski']
  }


  names = []
  train_scores = []
  test_scores = []

  for name, model in models.items():
    if name=='AdaBoosCls':
      param_grid=AdaBoosCls_param_grid
    elif name=='GradBoosCls':
      param_grid=GradBoosCls_param_grid
    elif name=='RanForCls':
      param_grid=RanForCls_param_grid
    elif name=='XGBoost':
      param_grid=XGBoost_param_grid
    elif name=='KNN':
      param_grid=KNN_param_grid
    elif name=='LogReg':
      param_grid=LogReg_param_grid
    # elif name=='Multinomial NB':
    #   param_grid=MNB_grid_params

    train_score,test_score,best_parameters=gridsearchcv(model,param_grid,X_train,X_test,y_train,y_test)
    names.append(name)
    train_scores.append(train_score) # Appending the test scores to the list
    test_scores.append(test_score) # Appending the test scores to the list
    result_tuned_df= pd.DataFrame({'model': names, 'train best score': train_scores, 'test best score': test_scores })
    print(' model: '+ name + '\n best parameters: ' )
    print( best_parameters)

  return result_tuned_df

##Function for ANN model

def NN_Model(X_train, X_test, y_train, y_test):
  in_dim = X_train.shape[1]

  tf.random.set_seed(7)

  model = Sequential()
  model.add(Dense(150, activation='relu', input_dim = in_dim))
  model.add(Dropout(0.2))
  model.add(Dense(50, activation='relu'))
  model.add(Dense(5 , activation='softmax'))

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=0, patience=3)

  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

  print(model.summary())

  history=model.fit(X_train, y_train_cat, validation_split=0.2, epochs = 100, batch_size = 50, callbacks=[early_stopping])

  train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  result_kfold_df= pd.DataFrame({'model': ['Neural Network'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })

  return result_kfold_df

# Neural network model

def build_clf(unit):
  # creating the layers of the NN
  tf.random.set_seed(7)
  ann = Sequential()
  ann.add(Dense(units=unit, activation='relu'))
  ann.add(Dense(units=unit, activation='relu'))
  ann.add(Dense(units=5, activation='softmax'))
  ann.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])
  return ann

## Function for Tuned ANN model



# from scikeras.wrappers import KerasClassifier

def Tuned_ANN(X,y):

  tf.random.set_seed(7)

  model=KerasClassifier(build_fn=build_clf)
  params={'batch_size':[100, 20, 50],
        'nb_epoch':[20, 50],
        'unit':[50,100,200,300],
        }
  gs=GridSearchCV(estimator=model, param_grid=params, cv=10)
  # now fit the dataset to the GridSearchCV object.
  y_cat=to_categorical(y)
  gs = gs.fit(X, y_cat)

  best_params=gs.best_params_
  accuracy=gs.best_score_

  print('best parameters for ANN: ' , best_params)
  print('best score for ANN: ' + str(accuracy))

def NN_Model_Tuned(X_train, X_test, y_train, y_test):
  in_dim = X_train.shape[1]

  tf.random.set_seed(7)

  model = Sequential()
  model.add(Dense(100, activation='relu', input_dim = in_dim))
  model.add(Dropout(0.2))
  model.add(Dense(100, activation='relu'))
  model.add(Dropout(0.2))
  model.add(Dense(50, activation='relu'))
  model.add(Dropout(0.2))
  model.add(Dense(50, activation='relu'))
  model.add(Dense(5 , activation='softmax'))

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=0, patience=3)

  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

  print(model.summary())

  history=model.fit(X_train, y_train_cat, validation_split=0.2, epochs = 20, batch_size = 50, callbacks=[early_stopping])

  train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  result_kfold_df= pd.DataFrame({'model': ['Neural Network'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })



  hist= pd.DataFrame(history.history)

  for col in hist.columns:
    print(col)
    plt.plot(hist[col])
    plt.plot(hist[col])
    plt.title('model-'+col)
    plt.ylabel(col)
    plt.xlabel('epoch')
    plt.show()

  return result_kfold_df

##Function for LSTM model

def LSTM_Model(X_train, X_test, y_train, y_test):
  in_dim = X_train.shape[1]

  tf.random.set_seed(7)
  embedding_vecor_length = 16

  model = Sequential()
  model.add(Embedding(max_features, embedding_vecor_length, input_length=in_dim))
  model.add(SpatialDropout1D(0.2))
  model.add(LSTM(200,dropout = 0.2, recurrent_dropout = 0.2))
  model.add(Dense(5 , activation='softmax'))

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=0, patience=3)

  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

  print(model.summary())

  history=model.fit(X_train, y_train_cat, validation_split=0.2, epochs = 100, batch_size = 20, callbacks=[early_stopping])

  train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  result_kfold_df= pd.DataFrame({'model': ['Neural Network'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })



  return result_kfold_df

# LSTM network model

def build_flstm(neurons):
  # creating the layers of the NN
  tf.random.set_seed(7)
  embedding_vecor_length=32
  model = Sequential()
  model.add(Embedding(max_features, embedding_vecor_length, input_length=maxlen))
  model.add(LSTM(units=neurons, activation='relu'))
  model.add(Dense(5 , activation='softmax'))
  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])
  return model

## Grid Search method to find tuned parameters for LSTM model

def Tuned_LSTM(X,y):

  tf.random.set_seed(7)

  model=KerasClassifier(build_fn=build_flstm)
  params={'neurons': [8, 16, 32, 64, 128, 256],
          'batch_size':[100, 20, 50],
        'nb_epoch':[20, 50],
        }
  gs=GridSearchCV(estimator=model, param_grid=params, cv=10)
  # now fit the dataset to the GridSearchCV object.
  y_cat=to_categorical(y)
  gs = gs.fit(X, y_cat)

  best_params=gs.best_params_
  accuracy=gs.best_score_

  print('best parameters for ANN: ' , best_params)
  print('best score for ANN: ' + str(accuracy))

## LSTM model after tuning the parameters

def LSTM_Model_Tuned(X_train, X_test, y_train, y_test):
  in_dim = X_train.shape[1]

  tf.random.set_seed(7)
  embedding_vecor_length = 32

  model = Sequential()
  model.add(Embedding(max_features, embedding_vecor_length, input_length=maxlen))
  model.add(SpatialDropout1D(0.2))
  model.add(LSTM(256))
  model.add(Dense(5 , activation='softmax'))

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=0, patience=3)

  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

  print(model.summary())

  history=model.fit(X_train, y_train_cat, validation_split=0.2, epochs = 50, batch_size = 100, callbacks=[early_stopping])

  train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  result_kfold_df= pd.DataFrame({'model': ['LSTM'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })

  hist= pd.DataFrame(history.history)

  for col in hist.columns:
    print(col)
    plt.plot(hist[col])
    plt.plot(hist[col])
    plt.title('model-'+col)
    plt.ylabel(col)
    plt.xlabel('epoch')
    plt.show()

  return result_kfold_df

## Function to implement Bi-directional model

def BI_LSTM_Model(X_train, X_test, y_train, y_test):
  in_dim = X_train.shape[1]

  tf.random.set_seed(7)
  embedding_vecor_length = 16

  model = Sequential()
  model.add(Embedding(max_features, embedding_vecor_length, input_length=in_dim))
  model.add(SpatialDropout1D(0.2))
  model.add(Bidirectional(LSTM(200,dropout = 0.2, recurrent_dropout = 0.2)))
  model.add(Dense(5 , activation='softmax'))

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=0, patience=3)

  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

  print(model.summary())

  history=model.fit(X_train, y_train_cat, validation_split=0.2, epochs = 100, batch_size = 20, callbacks=[early_stopping])

  train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  result_kfold_df= pd.DataFrame({'model': ['LSTM'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })


  return result_kfold_df

## Bi-directional LSTM model

def build_fbilstm(neurons):

  tf.random.set_seed(7)
  embedding_vecor_length=32
  model = Sequential()
  model.add(Embedding(max_features, embedding_vecor_length, input_length=maxlen))
  model.add(Bidirectional(LSTM(units=neurons, activation='relu')))
  model.add(Dense(5 , activation='softmax'))
  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])
  return model

## Grid Search method to find tuned parameters for Bi-directional LSTM model

def Tuned_BI_LSTM(X,y):

  tf.random.set_seed(7)

  model=KerasClassifier(build_fn=build_fbilstm)
  params={'neurons': [64, 128, 256],
          'batch_size':[100, 20, 50],
        'nb_epoch':[20, 50],
        }
  gs=GridSearchCV(estimator=model, param_grid=params, cv=10)
  # now fit the dataset to the GridSearchCV object.
  y_cat=to_categorical(y)
  gs = gs.fit(X, y_cat)

  best_params=gs.best_params_
  accuracy=gs.best_score_

  print('best parameters for ANN: ' , best_params)
  print('best score for ANN: ' + str(accuracy))

##Function for Birectional LSTM model with tuned parameters

def BI_LSTM_Model_Tuned(X_train, X_test, y_train, y_test):
  in_dim = X_train.shape[1]

  tf.random.set_seed(7)
  embedding_vecor_length = 16

  model = Sequential()
  model.add(Embedding(max_features, embedding_vecor_length, input_length=in_dim))
  model.add(SpatialDropout1D(0.2))
  model.add(Bidirectional(LSTM(128,dropout = 0.2, recurrent_dropout = 0.2)))
  model.add(Dense(5 , activation='softmax'))

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=0, patience=3)

  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

  print(model.summary())

  history=model.fit(X_train, y_train_cat, validation_split=0.2, epochs = 20, batch_size = 100, callbacks=[early_stopping])

  train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  result_kfold_df= pd.DataFrame({'model': ['LSTM'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })



  hist= pd.DataFrame(history.history)

  for col in hist.columns:
    print(col)
    plt.plot(hist[col])
    plt.plot(hist[col])
    plt.title('model-'+col)
    plt.ylabel(col)
    plt.xlabel('epoch')
    plt.show()

  return result_kfold_df

##Function for ANN model to be checked in Keras Tuner method

def build_keraslstm(hp):
  in_dim = X_train_cv.shape[1]
  tf.random.set_seed(7)
  embedding_vecor_length = 16
  model = Sequential()
  model.add(Embedding(max_features, embedding_vecor_length, input_length=maxlen))
  model.add(SpatialDropout1D(0.2))
  model.add(LSTM(units=hp.Int('units', min_value=32, max_value=512, step=32), activation='relu'))
  model.add(Dense(5 , activation='softmax'))
  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])
  return model

## Function to find best tuned parameters with kerastuner/randomsearch method

def Randomsearch_LSTM(X_train, X_test, y_train, y_test):

  tuner= RandomSearch(
          build_keraslstm,
          # objective='val_f1_score',
          objective='accuracy',
          max_trials=3,
          executions_per_trial=1,
          # direction='max',
          # objective=keras_tuner.Objective('val_f1_score', direction='max')
          )

  tuner.search_space_summary()

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  tuner.search(X_train,y_train_cat,epochs=5,validation_data=(X_test,y_test_cat))

  best_parameter= tuner.get_best_hyperparameters(1)[0]
  print('best parameters with Keras tuner method: ', best_parameter.values)

  best_model = tuner.get_best_models(num_models=1)
  print(best_model[0].summary())

  model=tuner.hypermodel.build(best_parameter)
  history=model.fit(X_train,y_train_cat,epochs=5,validation_data=(X_test,y_test_cat))

  model.compile(loss='categorical_crossentropy',optimizer='adam',metrics=['accuracy'])
  acc=accuracy_score(y_test_cat.argmax(axis=1),model.predict(X_test).argmax(axis=1))
  # f1_scr=f1_score(y_test_cat.argmax(axis=1),model.predict(X_test).argmax(axis=1))

  # train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  # test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  # train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  # test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  # result_kfold_df= pd.DataFrame({'model': ['LSTM'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })

  print('Random Search LSTM accuracy: ', acc)

  # return result_kfold_df

Importing the dataset

# Mount google drive to read file

from google.colab import drive
drive.mount('/content/drive/')

Mounted at /content/drive/

data = pd.read_csv('/content/drive/MyDrive/Colab Notebooks/Projects/NLP/IHMStefanini_industrial_safety_and_health_database_with_accidents_description.csv')

Checking basic features of dataset

# Checking the first 5 rows
data.head()

	Unnamed: 0	Data	Countries	Local	Industry Sector	Accident Level	Potential Accident Level	Genre	Employee or Third Party	Critical Risk	Description
0	0	2016-01-01 00:00:00	Country_01	Local_01	Mining	I	IV	Male	Third Party	Pressed	While removing the drill rod of the Jumbo 08 for maintenance, the supervisor proceeds to loosen the support of the intermediate centralizer to facilitate the removal, seeing this the mechanic supports one end on the drill of the equipment to pull with both hands the bar and accelerate the removal from this, at this moment the bar slides from its point of support and tightens the fingers of the mechanic between the drilling bar and the beam of the jumbo.
1	1	2016-01-02 00:00:00	Country_02	Local_02	Mining	I	IV	Male	Employee	Pressurized Systems	During the activation of a sodium sulphide pump, the piping was uncoupled and the sulfide solution was designed in the area to reach the maid. Immediately she made use of the emergency shower and was directed to the ambulatory doctor and later to the hospital. Note: of sulphide solution = 48 grams / liter.
2	2	2016-01-06 00:00:00	Country_01	Local_03	Mining	I	III	Male	Third Party (Remote)	Manual Tools	In the sub-station MILPO located at level +170 when the collaborator was doing the excavation work with a pick (hand tool), hitting a rock with the flat part of the beak, it bounces off hitting the steel tip of the safety shoe and then the metatarsal area of ​​the left foot of the collaborator causing the injury.
3	3	2016-01-08 00:00:00	Country_01	Local_04	Mining	I	I	Male	Third Party	Others	Being 9:45 am. approximately in the Nv. 1880 CX-695 OB7, the personnel begins the task of unlocking the Soquet bolts of the BHB machine, when they were in the penultimate bolt they identified that the hexagonal head was worn, proceeding Mr. Cristóbal - Auxiliary assistant to climb to the platform to exert pressure with your hand on the "DADO" key, to prevent it from coming out of the bolt; in those moments two collaborators rotate with the lever in anti-clockwise direction, leaving the key of the bolt, hitting the palm of the left hand, causing the injury.
4	4	2016-01-10 00:00:00	Country_01	Local_04	Mining	IV	IV	Male	Third Party	Others	Approximately at 11:45 a.m. in circumstances that the mechanics Anthony (group leader), Eduardo and Eric Fernández-injured-the three of the Company IMPROMEC, performed the removal of the pulley of the motor of the pump 3015 in the ZAF of Marcy. 27 cm / Length: 33 cm / Weight: 70 kg), as it was locked proceed to heating the pulley to loosen it, it comes out and falls from a distance of 1.06 meters high and hits the instep of the right foot of the worker, causing the injury described.

# Checking last 5 rows
data.tail()

	Unnamed: 0	Data	Countries	Local	Industry Sector	Accident Level	Potential Accident Level	Genre	Employee or Third Party	Critical Risk	Description
420	434	2017-07-04 00:00:00	Country_01	Local_04	Mining	I	III	Male	Third Party	Others	Being approximately 5:00 a.m. approximately, when lifting the Kelly HQ towards the pulley of the frame to align it, the assistant Marco that is in the later one is struck the hand against the frame generating the injury.
421	435	2017-07-04 00:00:00	Country_01	Local_03	Mining	I	II	Female	Employee	Others	The collaborator moved from the infrastructure office (Julio to the toilets, when the pin of the right shoe is hooked on the bra of the left shoe causing not to take the step and fall untimely, causing injury described.
422	436	2017-07-05 00:00:00	Country_02	Local_09	Metals	I	II	Male	Employee	Venomous Animals	During the environmental monitoring activity in the area, the employee was surprised by a swarming swarm of weevils. During the exit of the place, endured suffering two stings, being one in the face and the other in the middle finger of the left hand.
423	437	2017-07-06 00:00:00	Country_02	Local_05	Metals	I	II	Male	Employee	Cut	The Employee performed the activity of stripping cathodes, when pulling the cathode sheet his hand hit the side of another cathode, causing a blunt cut on his 2nd finger of the left hand.
424	438	2017-07-09 00:00:00	Country_01	Local_04	Mining	I	II	Female	Third Party	Fall prevention (same level)	At 10:00 a.m., when the assistant cleaned the floor of module "E" in the central camp, she slipped back and immediately grabbed the laundry table to avoid falling to the floor; suffering the described injury.

# Checking the dataset's dimensions
print(f"The dataset has {data.shape[0]} rows and {data.shape[1]} columns")

The dataset has 425 rows and 11 columns

# Summary dataset feature information
data.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 425 entries, 0 to 424
Data columns (total 11 columns):
 #   Column                    Non-Null Count  Dtype 
---  ------                    --------------  ----- 
 0   Unnamed: 0                425 non-null    int64 
 1   Data                      425 non-null    object
 2   Countries                 425 non-null    object
 3   Local                     425 non-null    object
 4   Industry Sector           425 non-null    object
 5   Accident Level            425 non-null    object
 6   Potential Accident Level  425 non-null    object
 7   Genre                     425 non-null    object
 8   Employee or Third Party   425 non-null    object
 9   Critical Risk             425 non-null    object
 10  Description               425 non-null    object
dtypes: int64(1), object(10)
memory usage: 36.6+ KB

# Checking index labels
data.index

RangeIndex(start=0, stop=425, step=1)

# Creating a list of all columns
data_columns = data.columns.tolist()
print(data_columns)

['Unnamed: 0', 'Data', 'Countries', 'Local', 'Industry Sector', 'Accident Level', 'Potential Accident Level', 'Genre', 'Employee or Third Party', 'Critical Risk', 'Description']

INITIAL OBSERVATIONS ON THE DATASET

1. The dataset has 425 rows and 11 columns

2. Columns

   a. 10 out of these 11 columns have data in a string format

   b.1 column, i.e. 'Unnamed: 0' has numerical integer data

   c. The 'Data' column has date and time based data but is in string format. This is to be converted into a datetime format

   d. The 'Unnamed: 0' column appears to be some kind of index. It is also not described in the Problem Statement. Likely candidate for deletion but to be assessed further before taking action

   e. The columns appear to be improperly named. (eg. 'Data' should be 'Date', 'Genre' should be 'Gender'... We'll rename these.

3. Rows

   a. There are 425 rows in the dataset, indexed 0:424

# Renaming the 'Data', 'Genre' & 'Employee or Third Party' columns

data = data.rename(columns = {"Data": "Date",
                              "Genre": "Gender",
                              "Employee or Third Party": "Personnel Type"})

# Changing datatype of 'Date' to datetime

data['Date'] = pd.to_datetime(data['Date'])

data['Date'].info()

<class 'pandas.core.series.Series'>
RangeIndex: 425 entries, 0 to 424
Series name: Date
Non-Null Count  Dtype         
--------------  -----         
425 non-null    datetime64[ns]
dtypes: datetime64[ns](1)
memory usage: 3.4 KB

# Checking the 'Unnamed: 0' column to determine the properties of its values

univariate_analysis(data, 'Unnamed: 0')

The column 'Unnamed: 0' has 425 unique values.

Unique values are:

[  0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17
  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35
  36  37  38  39  40  41  54  55  56  57  58  59  60  61  62  63  64  65
  66  67  68  69  70  71  72  73  74  75  76  77  78  79  80  81  82  83
  84  85  86  87  88  89  90  91  92  93  94  95  96  97  98  99 100 101
 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119
 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137
 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155
 156 157 158 159 160 161 162 163 164 165 166 169 170 171 172 173 174 175
 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193
 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211
 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229
 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247
 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265
 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283
 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301
 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319
 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337
 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355
 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373
 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391
 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409
 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427
 428 429 430 431 432 433 434 435 436 437 438]

The value counts of data in this column are:
0      1
306    1
304    1
303    1
302    1
      ..
151    1
150    1
149    1
148    1
438    1
Name: Unnamed: 0, Length: 425, dtype: int64

# Checking for missing sequential values in this column

number_range = list(range(data['Unnamed: 0'].values.max()))
unnamed0_values = data['Unnamed: 0'].tolist()
missing_numbers = []

for i in number_range:
    if i not in unnamed0_values:
        missing_numbers.append(i)

print(f"There are {len(missing_numbers)} missing numbers in the 'Unnamed: 0' column. They are\n")
print(missing_numbers)

There are 14 missing numbers in the 'Unnamed: 0' column. They are

[42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 167, 168]

Observations

Here we note that the 'Unnamed: 0' column has values in the range of 0 - 438, but has 14 missing numbers in the actual range of 0 - 438.

Our dataset has 425 row indices.

Given that we are not provided any information about this column in the dataset, and going by the column naming pattern, it is likely that 14 rows of an older version of this dataset, which originally had 439 rows, were deleted, and that modified dataset was then saved as the current csv file, with 425 rows.

Accordingly, we can delete this column

# Deleting the 'Unnamed: 0' column

data = data.drop('Unnamed: 0', axis = 1)

Treating Duplicate Values

# Checking for duplicates

data.duplicated().sum()

7

# Dropping duplicates from the dataset

data = data.drop_duplicates()

# Checking for duplicates again

data.duplicated().sum()

0

# Confirming absence of duplicates in each column, specifically looking for the Description column duplicates, since that column is critical to the NLP task

for i in data.columns:
    print(f"Column {i} has {data[i].duplicated().sum()} duplicate values")

Column Date has 131 duplicate values
Column Countries has 415 duplicate values
Column Local has 406 duplicate values
Column Industry Sector has 415 duplicate values
Column Accident Level has 413 duplicate values
Column Potential Accident Level has 412 duplicate values
Column Gender has 416 duplicate values
Column Personnel Type has 415 duplicate values
Column Critical Risk has 385 duplicate values
Column Description has 7 duplicate values

# Printing the duplicates in the 'Description' column

data[data.duplicated(subset=['Description'],keep=False)].sort_values(by='Description')

	Date	Countries	Local	Industry Sector	Accident Level	Potential Accident Level	Gender	Personnel Type	Critical Risk	Description
166	2016-07-07	Country_01	Local_03	Mining	IV	V	Male	Third Party	Others	At moments when the MAPERU truck of plate F1T 878, returned from the city of Pasco to the Unit transporting a consultant, being 350 meters from the main gate his lane is invaded by a civilian vehicle, making the driver turn sharply to the side right where was staff of the company IMPROMEC doing hot melt work in an 8 "pipe impacting two collaborators causing the injuries described At the time of the accident the truck was traveling at 37km / h - according to INTHINC -, the width of the road is of 6 meters, the activity had safety cones as a warning on both sides of the road and employees used their respective EPP'S.
167	2016-07-07	Country_01	Local_03	Mining	I	IV	Male	Third Party	Others	At moments when the MAPERU truck of plate F1T 878, returned from the city of Pasco to the Unit transporting a consultant, being 350 meters from the main gate his lane is invaded by a civilian vehicle, making the driver turn sharply to the side right where was staff of the company IMPROMEC doing hot melt work in an 8 "pipe impacting two collaborators causing the injuries described At the time of the accident the truck was traveling at 37km / h - according to INTHINC -, the width of the road is of 6 meters, the activity had safety cones as a warning on both sides of the road and employees used their respective EPP'S.
261	2016-12-01	Country_01	Local_03	Mining	I	IV	Male	Employee	Others	During the activity of chuteo of ore in hopper OP5; the operator of the locomotive parks his equipment under the hopper to fill the first car, it is at this moment that when it was blowing out to release the load, a mud flow suddenly appears with the presence of rock fragments; the personnel that was in the direction of the flow was covered with mud.
263	2016-12-01	Country_01	Local_03	Mining	I	IV	Male	Third Party	Others	During the activity of chuteo of ore in hopper OP5; the operator of the locomotive parks his equipment under the hopper to fill the first car, it is at this moment that when it was blowing out to release the load, a mud flow suddenly appears with the presence of rock fragments; the personnel that was in the direction of the flow was covered with mud.
412	2017-06-20	Country_01	Local_01	Mining	I	IV	Male	Employee	Others	In circumstance, the AHK-903 license plate (Empresa which serves the supervision of CMA, carried out the field inspections in the upper bank 4288, when unexpectedly when climbing through the operational access (positive ramp), It slides in an excavated area of ​​approximately 3 meters high, remaining in position with the front part on the floor. The occupants of the vehicle made use of the safety belt and all the complete epps.
413	2017-06-20	Country_01	Local_01	Mining	I	IV	Male	Third Party	Others	In circumstance, the AHK-903 license plate (Empresa which serves the supervision of CMA, carried out the field inspections in the upper bank 4288, when unexpectedly when climbing through the operational access (positive ramp), It slides in an excavated area of ​​approximately 3 meters high, remaining in position with the front part on the floor. The occupants of the vehicle made use of the safety belt and all the complete epps.
130	2016-05-26	Country_03	Local_10	Others	I	I	Male	Third Party	Bees	In the geological reconnaissance activity, in the farm of Mr. Lázaro, the team composed by Felipe and Divino de Morais, in normal activity encountered a ciliary forest, as they needed to enter the forest to verify a rock outcrop which was the front, the Divine realized the opening of the access with machete. At that moment, took a bite from his neck. There were no more attacks, no allergic reaction, and continued work normally. With the work completed, leaving the forest for the same access, the Divine assistant was attacked by a snake and suffered a sting in the forehead. At that moment they moved away from the area. It was verified that there was no type of allergic reaction and returned with normal activities.
131	2016-05-26	Country_03	Local_10	Others	I	I	Male	Employee	Others	In the geological reconnaissance activity, in the farm of Mr. Lázaro, the team composed by Felipe and Divino de Morais, in normal activity encountered a ciliary forest, as they needed to enter the forest to verify a rock outcrop which was the front, the Divine realized the opening of the access with machete. At that moment, took a bite from his neck. There were no more attacks, no allergic reaction, and continued work normally. With the work completed, leaving the forest for the same access, the Divine assistant was attacked by a snake and suffered a sting in the forehead. At that moment they moved away from the area. It was verified that there was no type of allergic reaction and returned with normal activities.
143	2016-06-08	Country_03	Local_10	Others	I	I	Male	Third Party	Bees	Project of Vazante that carried out sediment collection of current in the South of Mata target, in the drainage of Serra do Garrote. team that was composed of 04 members of the WCA company, being the S.r.s Leandro, and Jehovânio. they were moving from one collection point to another, inside a shallow drainage, they saw the bee carton, the reaction was to move away from the box as quickly as possible to avoid the stings, they ran about 50 meters, looking for a safe area, to exit the radius of attack of the bees, but the S.S. and Breno), were attacked and consequently they suffered 02 stings, in the belly and Jehovah in the hand, verified that there was no type of allergic reaction, returned with the normal activities.
144	2016-06-08	Country_03	Local_10	Others	I	I	Male	Third Party	Others	Project of Vazante that carried out sediment collection of current in the South of Mata target, in the drainage of Serra do Garrote. team that was composed of 04 members of the WCA company, being the S.r.s Leandro, and Jehovânio. they were moving from one collection point to another, inside a shallow drainage, they saw the bee carton, the reaction was to move away from the box as quickly as possible to avoid the stings, they ran about 50 meters, looking for a safe area, to exit the radius of attack of the bees, but the S.S. and Breno), were attacked and consequently they suffered 02 stings, in the belly and Jehovah in the hand, verified that there was no type of allergic reaction, returned with the normal activities.
387	2017-05-06	Country_02	Local_07	Mining	IV	V	Male	Employee	Projection	The employees Márcio and Sérgio performed the pump pipe clearing activity FZ1.031.4 and during the removal of the suction spool flange bolts, there was projection of pulp over them causing injuries.
388	2017-05-06	Country_02	Local_07	Mining	II	V	Male	Employee	Projection	The employees Márcio and Sérgio performed the pump pipe clearing activity FZ1.031.4 and during the removal of the suction spool flange bolts, there was projection of pulp over them causing injuries.
37	2016-02-24	Country_02	Local_07	Mining	I	V	Male	Employee	Others	When starting the activity of removing a coil of electric cables in the warehouse with the help of forklift truck the operator did not notice that there was a beehive in it. Due to the movement of the coil the bees were excited. Realizing the fact the operator turned off the equipment and left the area. People passing by were stung.
38	2016-02-24	Country_02	Local_07	Mining	I	V	Female	Third Party	Others	When starting the activity of removing a coil of electric cables in the warehouse with the help of forklift truck the operator did not notice that there was a beehive in it. Due to the movement of the coil the bees were excited. Realizing the fact the operator turned off the equipment and left the area. People passing by were stung.

# Dropping these duplicates (retaining one of the rows for each duplicate pair)

data.drop_duplicates(subset=['Description'], keep='first', inplace=True)

# Checking for duplicates in the 'Description' column again

data[data.duplicated(subset=['Description'],keep=False)].sort_values(by='Description')

	Date	Countries	Local	Industry Sector	Accident Level	Potential Accident Level	Gender	Personnel Type	Critical Risk	Description

data.duplicated().sum()

0

We note that all duplicates have been deleted from the dataset, specifically those pertaining to the 'Description' column

Checking for missing values

data.isna().sum()

Date                        0
Countries                   0
Local                       0
Industry Sector             0
Accident Level              0
Potential Accident Level    0
Gender                      0
Personnel Type              0
Critical Risk               0
Description                 0
dtype: int64

Observations

1. We note that there are no further duplicates, or missing values in the dataset

Resetting the indices of the dataframe

# Using reset_index

data = data.reset_index(drop = True)

UNIVARIATE ANALYSIS

1. Countries

univariate_analysis(data, 'Countries', normalize_data = False)

The column 'Countries' has 3 unique values.

Unique values are:

['Country_01' 'Country_02' 'Country_03']

The value counts of data in this column are:
Country_01    245
Country_02    127
Country_03     39
Name: Countries, dtype: int64

2. Local

univariate_analysis(data, 'Local', normalize_data = False)

The column 'Local' has 12 unique values.

Unique values are:

['Local_01' 'Local_02' 'Local_03' 'Local_04' 'Local_05' 'Local_06'
 'Local_07' 'Local_08' 'Local_10' 'Local_09' 'Local_11' 'Local_12']

The value counts of data in this column are:
Local_03    87
Local_05    59
Local_01    55
Local_04    55
Local_06    46
Local_10    39
Local_08    27
Local_02    23
Local_07    12
Local_12     4
Local_09     2
Local_11     2
Name: Local, dtype: int64

3. Industry - Sector

univariate_analysis(data, 'Industry Sector', normalize_data = False)

The column 'Industry Sector' has 3 unique values.

Unique values are:

['Mining' 'Metals' 'Others']

The value counts of data in this column are:
Mining    232
Metals    134
Others     45
Name: Industry Sector, dtype: int64

4. Gender

univariate_analysis(data, 'Gender', normalize_data = False)

The column 'Gender' has 2 unique values.

Unique values are:

['Male' 'Female']

The value counts of data in this column are:
Male      390
Female     21
Name: Gender, dtype: int64

5. Personnel Type

univariate_analysis(data, 'Personnel Type', normalize_data = False)

The column 'Personnel Type' has 3 unique values.

Unique values are:

['Third Party' 'Employee' 'Third Party (Remote)']

The value counts of data in this column are:
Third Party             180
Employee                176
Third Party (Remote)     55
Name: Personnel Type, dtype: int64

6. Critical Risk

univariate_analysis(data, 'Critical Risk', normalize_data = False)

The column 'Critical Risk' has 33 unique values.

Unique values are:

['Pressed' 'Pressurized Systems' 'Manual Tools' 'Others'
 'Fall prevention (same level)' 'Chemical substances' 'Liquid Metal'
 'Electrical installation' 'Confined space'
 'Pressurized Systems / Chemical Substances'
 'Blocking and isolation of energies' 'Suspended Loads' 'Poll' 'Cut'
 'Fall' 'Bees' 'Fall prevention' '\nNot applicable' 'Traffic' 'Projection'
 'Venomous Animals' 'Plates' 'Projection/Burning' 'remains of choco'
 'Vehicles and Mobile Equipment' 'Projection/Choco' 'Machine Protection'
 'Power lock' 'Burn' 'Projection/Manual Tools'
 'Individual protection equipment' 'Electrical Shock'
 'Projection of fragments']

The value counts of data in this column are:
Others                                       223
Pressed                                       24
Manual Tools                                  20
Chemical substances                           17
Cut                                           14
Venomous Animals                              13
Projection                                    12
Bees                                          10
Fall                                           9
Vehicles and Mobile Equipment                  8
Fall prevention (same level)                   7
remains of choco                               7
Pressurized Systems                            7
Fall prevention                                6
Suspended Loads                                6
Blocking and isolation of energies             3
Pressurized Systems / Chemical Substances      3
Power lock                                     3
Liquid Metal                                   3
Machine Protection                             2
Electrical Shock                               2
Poll                                           1
Individual protection equipment                1
Projection/Manual Tools                        1
Burn                                           1
Electrical installation                        1
Projection/Choco                               1
Projection/Burning                             1
Plates                                         1
Confined space                                 1
Traffic                                        1
\nNot applicable                               1
Projection of fragments                        1
Name: Critical Risk, dtype: int64

7. Description

• In order to process the 'Description' column, we will first cull it out into a separate Dataframe

data_description = pd.DataFrame(data['Description'])

# Checking the first 5 rows

data_description.head()

	Description
0	While removing the drill rod of the Jumbo 08 for maintenance, the supervisor proceeds to loosen the support of the intermediate centralizer to facilitate the removal, seeing this the mechanic supports one end on the drill of the equipment to pull with both hands the bar and accelerate the removal from this, at this moment the bar slides from its point of support and tightens the fingers of the mechanic between the drilling bar and the beam of the jumbo.
1	During the activation of a sodium sulphide pump, the piping was uncoupled and the sulfide solution was designed in the area to reach the maid. Immediately she made use of the emergency shower and was directed to the ambulatory doctor and later to the hospital. Note: of sulphide solution = 48 grams / liter.
2	In the sub-station MILPO located at level +170 when the collaborator was doing the excavation work with a pick (hand tool), hitting a rock with the flat part of the beak, it bounces off hitting the steel tip of the safety shoe and then the metatarsal area of ​​the left foot of the collaborator causing the injury.
3	Being 9:45 am. approximately in the Nv. 1880 CX-695 OB7, the personnel begins the task of unlocking the Soquet bolts of the BHB machine, when they were in the penultimate bolt they identified that the hexagonal head was worn, proceeding Mr. Cristóbal - Auxiliary assistant to climb to the platform to exert pressure with your hand on the "DADO" key, to prevent it from coming out of the bolt; in those moments two collaborators rotate with the lever in anti-clockwise direction, leaving the key of the bolt, hitting the palm of the left hand, causing the injury.
4	Approximately at 11:45 a.m. in circumstances that the mechanics Anthony (group leader), Eduardo and Eric Fernández-injured-the three of the Company IMPROMEC, performed the removal of the pulley of the motor of the pump 3015 in the ZAF of Marcy. 27 cm / Length: 33 cm / Weight: 70 kg), as it was locked proceed to heating the pulley to loosen it, it comes out and falls from a distance of 1.06 meters high and hits the instep of the right foot of the worker, causing the injury described.

# Using the 'sent_len' function created earlier to calculate the length of the text in each row

og_sent_len = sent_len(data_description, 'Description')

# Appending the list above to the 'data_description' dataframe
data_description['Original Sentence Length'] = og_sent_len

# Inspecting the dataframe
data_description.head()

	Description	Original Sentence Length
0	While removing the drill rod of the Jumbo 08 for maintenance, the supervisor proceeds to loosen the support of the intermediate centralizer to facilitate the removal, seeing this the mechanic supports one end on the drill of the equipment to pull with both hands the bar and accelerate the removal from this, at this moment the bar slides from its point of support and tightens the fingers of the mechanic between the drilling bar and the beam of the jumbo.	80
1	During the activation of a sodium sulphide pump, the piping was uncoupled and the sulfide solution was designed in the area to reach the maid. Immediately she made use of the emergency shower and was directed to the ambulatory doctor and later to the hospital. Note: of sulphide solution = 48 grams / liter.	54
2	In the sub-station MILPO located at level +170 when the collaborator was doing the excavation work with a pick (hand tool), hitting a rock with the flat part of the beak, it bounces off hitting the steel tip of the safety shoe and then the metatarsal area of ​​the left foot of the collaborator causing the injury.	57
3	Being 9:45 am. approximately in the Nv. 1880 CX-695 OB7, the personnel begins the task of unlocking the Soquet bolts of the BHB machine, when they were in the penultimate bolt they identified that the hexagonal head was worn, proceeding Mr. Cristóbal - Auxiliary assistant to climb to the platform to exert pressure with your hand on the "DADO" key, to prevent it from coming out of the bolt; in those moments two collaborators rotate with the lever in anti-clockwise direction, leaving the key of the bolt, hitting the palm of the left hand, causing the injury.	97
4	Approximately at 11:45 a.m. in circumstances that the mechanics Anthony (group leader), Eduardo and Eric Fernández-injured-the three of the Company IMPROMEC, performed the removal of the pulley of the motor of the pump 3015 in the ZAF of Marcy. 27 cm / Length: 33 cm / Weight: 70 kg), as it was locked proceed to heating the pulley to loosen it, it comes out and falls from a distance of 1.06 meters high and hits the instep of the right foot of the worker, causing the injury described.	88

# Checking the statistics of the Description column

data_description['Original Sentence Length'].describe()

count    411.000000
mean      64.717762
std       31.916479
min       16.000000
25%       40.000000
50%       60.000000
75%       82.500000
max      183.000000
Name: Original Sentence Length, dtype: float64

# Plotting the distribution of Original_Sentence Length

fig, ax = plt.subplots(nrows = 1, ncols = 1, figsize = (8, 6))
ax.set_title("Distribution of Lengths of Descriptions", fontsize = 15)
sns.histplot(data = data_description, x = 'Original Sentence Length', ax = ax, kde = True)
ax.set_xlabel("Number of words in Sentence")
plt.show()

# Checking the total word count across all documents

total_sentences_raw = ' '.join(sentence for sentence in data_description['Description'])
raw_wordcount = len(total_sentences_raw)
print(f"The total number of words in the raw 'Description' column are {raw_wordcount}.")

The total number of words in the raw 'Description' column are 149768.

8. ACCIDENT LEVEL

Potential Target Variable

univariate_analysis(data, 'Accident Level', normalize_data = False)

The column 'Accident Level' has 5 unique values.

Unique values are:

['I' 'IV' 'III' 'II' 'V']

The value counts of data in this column are:
I      303
II      39
III     31
IV      30
V        8
Name: Accident Level, dtype: int64

Observations

1. The target variable is highly skewed, with approx. 75% of the accidents being classified as Accident Level I

2. We will have to keep this imbalance in mind when preparing the dataset for supervised ML models. We could experiment with SMOTE here too.

3. The 'Potential Accident Level' column (below), which our guide has advised us as a potentially more appropriate target variable, is more balanced compared to this column. We could likely use that column instead for our models

4. We will also have to change the label values to int.

univariate_analysis(data, 'Potential Accident Level', normalize_data = False)

The column 'Potential Accident Level' has 6 unique values.

Unique values are:

['IV' 'III' 'I' 'II' 'V' 'VI']

The value counts of data in this column are:
IV     138
III    106
II      95
I       43
V       28
VI       1
Name: Potential Accident Level, dtype: int64

Observations

1. Unlike the 'Accident Level' column, here Level IV has the highest representation (34%), followed by Level III (25%), Level II (22%), Level I (12%) and Level V (7%).

2. Also unlike the 'Accident Level' column, there is an extra Potential Accident level - Level VI - in this column. However, there is only 1 instance of this Potential Accident Level.

3. We will merge VI with V while preparing the dataset for machine learning

4. We will have to change the label values from I - VI to 1 - 6

NOTE: Choice of Target Variable going forward

1. While the distribution of labels in the ‘Accident Level’ column may give the appearance of accidents being largely less serious (Level I is the lowest Accident Level), a comparison with their corresponding values in the ‘Potential Accident Level’ column indicate that these accidents could have been far more serious and severe than they actually were.

2. Given the nature of the task, the proposed chatbot would return a potential accident level, given a particular description. It would be more appropriate for such a chatbot to err on the side of conservatism while reporting the potential severity of an accident. This would help medical and other staff treat the accident with appropriate gravity.

3. The ‘Potential Accident Level’ column, while being more balanced in its distribution of accident level values, is also more conservative in its labeling of the severity levels of accidents, as compared to the ‘Accident Level’ column. Accordingly, we deduce that machine learning models would likely be more appropriately trained, from a business perspective, by using the ‘Potential Accident Level’ column as a target variable, rather than the ‘Accident Level’ column.

BIVARIATE ANALYSIS

1. Countries - Potential Accident Level

bivariate_analysis(dataframe = data, variable_1 = 'Countries', variable_2 = 'Potential Accident Level')

Observations

1. In terms of absolute representation, Country_01 has a higher number of observations, compared to Country_02 and Country_03.

2. We note that both Country_01 and Country_02 have a higher incidence of Potential Accident Level IV, III and II, compared to the other levels.

3. Country_03 has recorded the highest comparative incidences of Potential Accident Level I, compared to the other levels.

2. Local - Potential Accident Level

bivariate_analysis(dataframe = data, variable_1 = 'Local', variable_2 = 'Potential Accident Level')

Observations

1. Potential Accident Level IV is the most frequently occuring level in Locals 01, 03, 04 and 05.

2. Potential Accident Level I is the most frequently occuring level for Local_10, with the other levels occuring much more infrequently.

3. Local_09, Local_11 and Local_12 have the lowest number of observations in the dataset, with the first two only having recorded Potential Accident Levels of IV and II.

3. Industry Sector - Potential Accident Level

bivariate_analysis(dataframe = data, variable_1 = 'Industry Sector', variable_2 = 'Potential Accident Level')

Observations

1. The Mining Sector sees the highest incidences of Potential Accident Level IV, followed by Level II and Level II.

2. On the other hand, the Metals sector sees a somewhat comparable number of Potential Accident Levels II and III, with Level II being slightly greater than III. Level III in turn is closely followed by Level IV

3. All the other sectors see a greater frequency of occurences in Potential Level I, indicating that accidents in these sectors are likely more trivial compared to Mining and Metals.

4. Gender - Potential Accident Level

bivariate_analysis(dataframe = data, variable_1 = 'Gender', variable_2 = 'Potential Accident Level')

Observations

1. As observed in the univariate analysis, representation of Males is overwhelmingly imbalanced in the dataset.

2. It is observed that Males are most highly susceptible to Potential Accident Levels IV, III and II respectively, in descending order of their frequencies.

3. On the other hand, Females are more likely to suffer Potential Accident Levels II. There are no observed instances of Potential Accident Levels I, V or VI vis-a-vis Female personnel

5. Personnel Type - Potential Accident Level

bivariate_analysis(dataframe = data, variable_1 = 'Personnel Type', variable_2 = 'Potential Accident Level')

Observations

1. Third Party Personnel see the highest frequency of occurence in Potential Accident Level IV cases. This is followed by Levels III and II which are almost at par with each other, and Level I, which is only somewhat less frequent than III or II.

2. Employees also see a high incidence of Potential Accident Levels IV, III and II

3. Third Party (Remote) personnel also follow the same pattern as the other two types vis-a-vis Potential Accident Level distribution. However, at an absolute level, there are fewer observations of Third Party (Remote) workers compared to Employees and On-Site Third Party personnel, and thus their corresponding Potential Accident Level frequencies are correspondingly smaller compared to the other two types of personnel.

bivariate_analysis(dataframe = data, variable_1 = 'Critical Risk', variable_2 = 'Potential Accident Level')

Observations

1. The Critical Risk column does not offer us much information. While there are a wide array of feature values in the column, their frequency of appearance is relatively much lower compared to critical risks classified as ‘Others’. (‘Others’ constitutes nearly ~54.3% of the total values in the column, with 32 other types of critical risk making up the rest of the values.)

2. Since ‘Others’ does not offer us any further information, we do not perform a bivariate analysis of the Critical Risk column against the target variable, since that would not yield us any meaningful interpretable results.

DATA PRE-PROCESSING

# Previewing the dataframe

data.head(n = 20)

	Date	Countries	Local	Industry Sector	Accident Level	Potential Accident Level	Gender	Personnel Type	Critical Risk	Description
0	2016-01-01	Country_01	Local_01	Mining	I	IV	Male	Third Party	Pressed	While removing the drill rod of the Jumbo 08 for maintenance, the supervisor proceeds to loosen the support of the intermediate centralizer to facilitate the removal, seeing this the mechanic supports one end on the drill of the equipment to pull with both hands the bar and accelerate the removal from this, at this moment the bar slides from its point of support and tightens the fingers of the mechanic between the drilling bar and the beam of the jumbo.
1	2016-01-02	Country_02	Local_02	Mining	I	IV	Male	Employee	Pressurized Systems	During the activation of a sodium sulphide pump, the piping was uncoupled and the sulfide solution was designed in the area to reach the maid. Immediately she made use of the emergency shower and was directed to the ambulatory doctor and later to the hospital. Note: of sulphide solution = 48 grams / liter.
2	2016-01-06	Country_01	Local_03	Mining	I	III	Male	Third Party (Remote)	Manual Tools	In the sub-station MILPO located at level +170 when the collaborator was doing the excavation work with a pick (hand tool), hitting a rock with the flat part of the beak, it bounces off hitting the steel tip of the safety shoe and then the metatarsal area of ​​the left foot of the collaborator causing the injury.
3	2016-01-08	Country_01	Local_04	Mining	I	I	Male	Third Party	Others	Being 9:45 am. approximately in the Nv. 1880 CX-695 OB7, the personnel begins the task of unlocking the Soquet bolts of the BHB machine, when they were in the penultimate bolt they identified that the hexagonal head was worn, proceeding Mr. Cristóbal - Auxiliary assistant to climb to the platform to exert pressure with your hand on the "DADO" key, to prevent it from coming out of the bolt; in those moments two collaborators rotate with the lever in anti-clockwise direction, leaving the key of the bolt, hitting the palm of the left hand, causing the injury.
4	2016-01-10	Country_01	Local_04	Mining	IV	IV	Male	Third Party	Others	Approximately at 11:45 a.m. in circumstances that the mechanics Anthony (group leader), Eduardo and Eric Fernández-injured-the three of the Company IMPROMEC, performed the removal of the pulley of the motor of the pump 3015 in the ZAF of Marcy. 27 cm / Length: 33 cm / Weight: 70 kg), as it was locked proceed to heating the pulley to loosen it, it comes out and falls from a distance of 1.06 meters high and hits the instep of the right foot of the worker, causing the injury described.
5	2016-01-12	Country_02	Local_05	Metals	I	III	Male	Third Party (Remote)	Pressurized Systems	During the unloading operation of the ustulado Bag there was a need to unclog the discharge mouth of the silo truck. In performing this procedure, there was a maneuver of unhooking the hose without the total depressurisation of the mouth, projecting ustulado powder in the collaborator caused irritation in the eyes.
6	2016-01-16	Country_02	Local_05	Metals	I	III	Male	Employee	Fall prevention (same level)	The collaborator reports that he was on street 09 holding in his left hand the volumetric balloon, when he slipped and when placing his hand on the ground the volumetric balloon ended up breaking caused a small wound in his left hand.
7	2016-01-17	Country_01	Local_04	Mining	I	III	Male	Third Party	Pressed	At approximately 04:50 p.m., when the mechanic technician José of the Tecnomin verified the transmission belts of the HM-100 pump at the Acid plant, he proceeded to turn the pulley manually; unexpectedly at that instant the electrician supervisor Miguel of the EKA Mining grabs the transmission belts to verify their tension, at which point the finger traps.
8	2016-01-19	Country_02	Local_02	Mining	I	IV	Male	Third Party (Remote)	Others	Employee was sitting in the resting area at level 326 (raise bore), when he suffered sudden illness, falling and suffering excoriation on the face.
9	2016-01-26	Country_01	Local_06	Metals	I	II	Male	Third Party	Chemical substances	At the moment the forklift operator went to manipulate big bag of bioxide in section 70 and just in front of the ladder that leads to the area of ​​manual displacement, he splashed spent at the height of his forehead from a fissure in pipe G -069, subsequently spilling to his left eye. The collaborator went to the nearby eyewash for cleaning and immediately to the medical center.
10	2016-01-28	Country_01	Local_03	Mining	I	III	Male	Employee	Others	While installing a segment of the polyurethane pulley protective lyner - 60x4x5cm weighing 1.2 kg - on the head pulley of the ore winch, when the pulley is rotated to compress the lyner inside the channel, it falls from its housing 1.50 m rubbing the right side of the worker's hip, generating the injury described.
11	2016-01-30	Country_01	Local_03	Mining	I	IV	Male	Third Party	Others	While preparing the rice for the lunch of the day, when moving the pot # 60 - 35 Kg of weight including the contents - to evacuate the residual water of the cooking of the rice, when positioning the pot on a jaba it tilts backwards spilling some 200 ml of hot water on the cook's leg. The cook immediately after the event applies first aid, pouring cold water on the area of ​​the injury and go to the medical post for evaluation.
12	2016-02-01	Country_02	Local_05	Metals	I	I	Male	Employee	Liquid Metal	The collaborator reports that he was working in the Ustulación and realized that the cyclone duct was obstructed and opened the door to try to unclog the same and the material detached and projected towards the employee causing small burn in the right heel.
13	2016-02-02	Country_01	Local_01	Mining	IV	V	Male	Third Party	Electrical installation	In moments that the operator of the Jumbo 2, tried energize your equipment to proceed to the installation of 4 split set at intersection 544 of Nv 3300, remove the lock and opening the electric board of 440V and 400A, and when lifting the thermomagnetic key This makes phase to ground - phase contact with the panel shell - producing a flash which reaches the operator causing the injury described.
14	2016-02-04	Country_02	Local_05	Metals	I	III	Male	Employee	Confined space	Due to the accumulation of Waelz on the conveyor and trailer of the Filter 08FI0502, the employee performed the cleaning of the shutter using the air lance, when he was surprised by the fall of the product that was above the door, passing between the neck and the collar of the aramid jacket and causing burn in the neck and shoulder.
15	2016-02-04	Country_02	Local_05	Metals	I	IV	Male	Employee	Liquid Metal	The employee was working in the When a thermal shock caused a splash of zinc in his direction, the employee, despite using all the indicated PPE, was hit by small spatters that passed between the facila and the hood. small burn in the face region.
16	2016-02-06	Country_01	Local_04	Mining	III	IV	Male	Third Party	Others	At Rp 050 of level 1620, in circumstances where the workers of the company were performing the task of diamond drilling, the assistants Jhonatan (injured) and Nilton were preparing to increase the HQ perforation pipe located on the scaffolding, Mr. Jhonatan lifts one end of the tube and supports it on the pulley of the equipment frame, the other end being on the working scaffolding, at the moment that Mr. Nilton lifts the end of the HQ pipe that is in the scaffolding to position in the frame, the upper part of the pipe comes out of the pulley falling and striking the right hand of the worker jhonatan against bolts that has the lateral part of the same frame causing the injury described.
17	2016-02-07	Country_01	Local_06	Metals	I	II	Female	Third Party	Others	Due to the overheating of 2 bars in row 5 of cell 7 a spark is produced, which is projected and manages to reach the Chief of guard who was in the corridor, producing a first degree burn in the neck.
18	2016-02-08	Country_01	Local_06	Metals	I	II	Male	Employee	Others	An auxiliary wheel of the cathode crane G2133 was changed in area 75, when when a bearing was heated and hit with a hammer and chisel at one end of the bearing track, a detachment of a bearing piece occurred, impacting it in the thigh of the right leg producing a cut. The ambulance is called and you are transferred to the clinic.
19	2016-02-21	Country_01	Local_06	Metals	I	III	Male	Employee	Others	The worker Manuel was making the disconnection of the power cables of the gate that is at the intersection of Manco streets with Cajamarquilla in order to remove it. In circumstances that Mr. José worker of the company ITS, was removing the rope tied in the body of the gate, this yields and falls pulling the warning post which hits the helmet of Mr. who He was standing at his side.

1. Pre-Processing the 'Description' column

a. We will use the 'clean_text' function and apply it to the descriptions culled out earlier in the 'data_descriptions' column. The cleaned data will be stored in a new variable in the data_description DataFrame, named 'Cleaned Description'

b. We then append the 'Cleaned Description' column to the original 'data' DataFrame

# Cleaning the 'Description' column in 'data_description'

data_description['Cleaned Description'] = data_description['Description'].apply(clean_text)

# Inspecting the data_description DataFrame

data_description.head(n = 10)

	Description	Original Sentence Length	Cleaned Description
0	While removing the drill rod of the Jumbo 08 for maintenance, the supervisor proceeds to loosen the support of the intermediate centralizer to facilitate the removal, seeing this the mechanic supports one end on the drill of the equipment to pull with both hands the bar and accelerate the removal from this, at this moment the bar slides from its point of support and tightens the fingers of the mechanic between the drilling bar and the beam of the jumbo.	80	while removing drill rod jumbo maintenance supervisor proceeds loosen support intermediate centralizer facilitate removal seeing mechanic support one end drill equipment pull hand bar accelerate removal moment bar slide point support tightens finger mechanic between drilling bar beam jumbo
1	During the activation of a sodium sulphide pump, the piping was uncoupled and the sulfide solution was designed in the area to reach the maid. Immediately she made use of the emergency shower and was directed to the ambulatory doctor and later to the hospital. Note: of sulphide solution = 48 grams / liter.	54	during activation sodium sulphide pump piping uncoupled sulfide solution designed area reach maid immediately made use emergency shower directed ambulatory doctor later hospital note sulphide solution gram liter
2	In the sub-station MILPO located at level +170 when the collaborator was doing the excavation work with a pick (hand tool), hitting a rock with the flat part of the beak, it bounces off hitting the steel tip of the safety shoe and then the metatarsal area of ​​the left foot of the collaborator causing the injury.	57	substation milpo located level when collaborator excavation work pick hand tool hitting rock flat part beak bounce hitting steel tip safety shoe metatarsal area left foot collaborator causing injury
3	Being 9:45 am. approximately in the Nv. 1880 CX-695 OB7, the personnel begins the task of unlocking the Soquet bolts of the BHB machine, when they were in the penultimate bolt they identified that the hexagonal head was worn, proceeding Mr. Cristóbal - Auxiliary assistant to climb to the platform to exert pressure with your hand on the "DADO" key, to prevent it from coming out of the bolt; in those moments two collaborators rotate with the lever in anti-clockwise direction, leaving the key of the bolt, hitting the palm of the left hand, causing the injury.	97	approximately nv personnel begin task unlocking soquet bolt bhb machine when penultimate bolt identified hexagonal head worn proceeding mr cristóbal auxiliary assistant climb platform exert pressure hand dado key prevent coming bolt moment two collaborator rotate lever anticlockwise direction leaving key bolt hitting palm left hand causing injury
4	Approximately at 11:45 a.m. in circumstances that the mechanics Anthony (group leader), Eduardo and Eric Fernández-injured-the three of the Company IMPROMEC, performed the removal of the pulley of the motor of the pump 3015 in the ZAF of Marcy. 27 cm / Length: 33 cm / Weight: 70 kg), as it was locked proceed to heating the pulley to loosen it, it comes out and falls from a distance of 1.06 meters high and hits the instep of the right foot of the worker, causing the injury described.	88	approximately circumstance mechanic anthony group leader eduardo eric fernándezinjuredthe three company impromec performed removal pulley motor pump zaf marcy cm length cm weight kg locked proceed heating pulley loosen come fall distance meter high hit instep right foot worker causing injury described
5	During the unloading operation of the ustulado Bag there was a need to unclog the discharge mouth of the silo truck. In performing this procedure, there was a maneuver of unhooking the hose without the total depressurisation of the mouth, projecting ustulado powder in the collaborator caused irritation in the eyes.	51	during unloading operation ustulado bag need unclog discharge mouth silo truck performing procedure maneuver unhooking hose without total depressurisation mouth projecting ustulado powder collaborator caused irritation eye
6	The collaborator reports that he was on street 09 holding in his left hand the volumetric balloon, when he slipped and when placing his hand on the ground the volumetric balloon ended up breaking caused a small wound in his left hand.	42	collaborator report street holding left hand volumetric balloon when slipped when placing hand ground volumetric balloon ended breaking caused small wound left hand
7	At approximately 04:50 p.m., when the mechanic technician José of the Tecnomin verified the transmission belts of the HM-100 pump at the Acid plant, he proceeded to turn the pulley manually; unexpectedly at that instant the electrician supervisor Miguel of the EKA Mining grabs the transmission belts to verify their tension, at which point the finger traps.	57	approximately pm when mechanic technician josé tecnomin verified transmission belt pump acid plant proceeded turn pulley manually unexpectedly instant electrician supervisor miguel eka mining grab transmission belt verify tension point finger trap
8	Employee was sitting in the resting area at level 326 (raise bore), when he suffered sudden illness, falling and suffering excoriation on the face.	24	employee sitting resting area level raise bore when suffered sudden illness falling suffering excoriation face
9	At the moment the forklift operator went to manipulate big bag of bioxide in section 70 and just in front of the ladder that leads to the area of ​​manual displacement, he splashed spent at the height of his forehead from a fissure in pipe G -069, subsequently spilling to his left eye. The collaborator went to the nearby eyewash for cleaning and immediately to the medical center.	68	moment forklift operator went manipulate big bag bioxide section front ladder lead area manual displacement splashed spent height forehead fissure pipe subsequently spilling left eye collaborator went nearby eyewash cleaning immediately medical center

# Using the 'sent_len' function created earlier to calculate the length of the text in each row

clean_sent_len = sent_len(data_description, 'Cleaned Description')

# Appending the list above to the 'data_description' dataframe
data_description['Cleaned Sentence Length'] = clean_sent_len

# Inspecting the dataframe
data_description.head()

	Description	Original Sentence Length	Cleaned Description	Cleaned Sentence Length
0	While removing the drill rod of the Jumbo 08 for maintenance, the supervisor proceeds to loosen the support of the intermediate centralizer to facilitate the removal, seeing this the mechanic supports one end on the drill of the equipment to pull with both hands the bar and accelerate the removal from this, at this moment the bar slides from its point of support and tightens the fingers of the mechanic between the drilling bar and the beam of the jumbo.	80	while removing drill rod jumbo maintenance supervisor proceeds loosen support intermediate centralizer facilitate removal seeing mechanic support one end drill equipment pull hand bar accelerate removal moment bar slide point support tightens finger mechanic between drilling bar beam jumbo	39
1	During the activation of a sodium sulphide pump, the piping was uncoupled and the sulfide solution was designed in the area to reach the maid. Immediately she made use of the emergency shower and was directed to the ambulatory doctor and later to the hospital. Note: of sulphide solution = 48 grams / liter.	54	during activation sodium sulphide pump piping uncoupled sulfide solution designed area reach maid immediately made use emergency shower directed ambulatory doctor later hospital note sulphide solution gram liter	28
2	In the sub-station MILPO located at level +170 when the collaborator was doing the excavation work with a pick (hand tool), hitting a rock with the flat part of the beak, it bounces off hitting the steel tip of the safety shoe and then the metatarsal area of ​​the left foot of the collaborator causing the injury.	57	substation milpo located level when collaborator excavation work pick hand tool hitting rock flat part beak bounce hitting steel tip safety shoe metatarsal area left foot collaborator causing injury	29
3	Being 9:45 am. approximately in the Nv. 1880 CX-695 OB7, the personnel begins the task of unlocking the Soquet bolts of the BHB machine, when they were in the penultimate bolt they identified that the hexagonal head was worn, proceeding Mr. Cristóbal - Auxiliary assistant to climb to the platform to exert pressure with your hand on the "DADO" key, to prevent it from coming out of the bolt; in those moments two collaborators rotate with the lever in anti-clockwise direction, leaving the key of the bolt, hitting the palm of the left hand, causing the injury.	97	approximately nv personnel begin task unlocking soquet bolt bhb machine when penultimate bolt identified hexagonal head worn proceeding mr cristóbal auxiliary assistant climb platform exert pressure hand dado key prevent coming bolt moment two collaborator rotate lever anticlockwise direction leaving key bolt hitting palm left hand causing injury	48
4	Approximately at 11:45 a.m. in circumstances that the mechanics Anthony (group leader), Eduardo and Eric Fernández-injured-the three of the Company IMPROMEC, performed the removal of the pulley of the motor of the pump 3015 in the ZAF of Marcy. 27 cm / Length: 33 cm / Weight: 70 kg), as it was locked proceed to heating the pulley to loosen it, it comes out and falls from a distance of 1.06 meters high and hits the instep of the right foot of the worker, causing the injury described.	88	approximately circumstance mechanic anthony group leader eduardo eric fernándezinjuredthe three company impromec performed removal pulley motor pump zaf marcy cm length cm weight kg locked proceed heating pulley loosen come fall distance meter high hit instep right foot worker causing injury described	42

# Checking the statistics of the Cleaned Description column

data_description['Cleaned Sentence Length'].describe()

count    411.000000
mean      33.238443
std       15.702309
min        9.000000
25%       21.500000
50%       31.000000
75%       42.000000
max       96.000000
Name: Cleaned Sentence Length, dtype: float64

# Plotting the distribution of Cleaned Sentence Length

fig, ax = plt.subplots(nrows = 1, ncols = 1, figsize = (8, 6))
ax.set_title("Distribution of Lengths of Cleaned Descriptions", fontsize = 15)
sns.histplot(data = data_description, x = 'Cleaned Sentence Length', ax = ax, kde = True)
ax.set_xlabel("Number of words in each document(cleaned)")
plt.show()

# Checking the total word count across all documents

total_sentences_clean = ' '.join(sentence for sentence in data_description['Cleaned Description'])
clean_wordcount = len(total_sentences_clean)
print(f"The total number of words in the cleaned 'Description' column are {clean_wordcount}.")

The total number of words in the cleaned 'Description' column are 96977.

# Appending 'Cleaned Description' to the main 'data' DataFrame

data['Cleaned Description'] = data_description['Cleaned Description']

2. Pre-Processing the other features

# Dropping the 'Date' column
data = data.drop('Date', axis = 1)

# Dropping the 'Accident Level' column
data = data.drop('Accident Level', axis = 1)

# Renaming values in the 'Critical Risk' column
data['Critical Risk'] = data['Critical Risk'].apply(lambda x: 'Uncategorized' if x == 'Others' else 'Categorized')

# Dropping the 'Description' column
data = data.drop('Description', axis = 1)

# Clubbing the label 'VI' in 'Potential Accident Level', with 'V'
data['Potential Accident Level'] = data['Potential Accident Level'].apply(lambda x: 'V' if x == 'VI' else x)

# Renaming values in the 'Potential Accident Level' column
le = LabelEncoder()
data['Potential Accident Level'] = le.fit_transform(data['Potential Accident Level'])

# Creating dummy variables for Countries, Local, Industry Sector, Gender, Personnel Type and Critical Risk
dummy_columns = ['Countries', 'Local', 'Industry Sector', 'Gender', 'Personnel Type', 'Critical Risk']
data = pd.get_dummies(data, columns = dummy_columns, drop_first = True)

# Preview the pre-processed dataset
data.head()

	Potential Accident Level	Cleaned Description	Countries_Country_02	Countries_Country_03	Local_Local_02	Local_Local_03	Local_Local_04	Local_Local_05	Local_Local_06	Local_Local_07	...	Local_Local_09	Local_Local_10	Local_Local_11	Local_Local_12	Industry Sector_Mining	Industry Sector_Others	Gender_Male	Personnel Type_Third Party	Personnel Type_Third Party (Remote)	Critical Risk_Uncategorized
0	3	while removing drill rod jumbo maintenance supervisor proceeds loosen support intermediate centralizer facilitate removal seeing mechanic support one end drill equipment pull hand bar accelerate removal moment bar slide point support tightens finger mechanic between drilling bar beam jumbo	0	0	0	0	0	0	0	0	...	0	0	0	0	1	0	1	1	0	0
1	3	during activation sodium sulphide pump piping uncoupled sulfide solution designed area reach maid immediately made use emergency shower directed ambulatory doctor later hospital note sulphide solution gram liter	1	0	1	0	0	0	0	0	...	0	0	0	0	1	0	1	0	0	0
2	2	substation milpo located level when collaborator excavation work pick hand tool hitting rock flat part beak bounce hitting steel tip safety shoe metatarsal area left foot collaborator causing injury	0	0	0	1	0	0	0	0	...	0	0	0	0	1	0	1	0	1	0
3	0	approximately nv personnel begin task unlocking soquet bolt bhb machine when penultimate bolt identified hexagonal head worn proceeding mr cristóbal auxiliary assistant climb platform exert pressure hand dado key prevent coming bolt moment two collaborator rotate lever anticlockwise direction leaving key bolt hitting palm left hand causing injury	0	0	0	0	1	0	0	0	...	0	0	0	0	1	0	1	1	0	1
4	3	approximately circumstance mechanic anthony group leader eduardo eric fernándezinjuredthe three company impromec performed removal pulley motor pump zaf marcy cm length cm weight kg locked proceed heating pulley loosen come fall distance meter high hit instep right foot worker causing injury described	0	0	0	0	1	0	0	0	...	0	0	0	0	1	0	1	1	0	1

5 rows × 21 columns

WORDCLOUDS

# Creating a wordcloud of cleaned descriptions labels with Potential Accident Level I

label_I = data[data['Potential Accident Level'] == 0]
all_labelI_words = ' '.join(label_I['Cleaned Description'])
joined_labelI_words = " ".join([word for word in all_labelI_words.split()])

label_I_wordcloud = WordCloud(background_color = 'black', width = 3000, height = 2500).generate(joined_labelI_words)

plt.figure(1, figsize = (12, 12))
plt.imshow(label_I_wordcloud)
plt.axis('off')
plt.show()

# Creating a wordcloud of cleaned descriptions labels with Potential Accident Level II

label_II = data[data['Potential Accident Level'] == 1]
all_labelII_words = ' '.join(label_II['Cleaned Description'])
joined_labelII_words = " ".join([word for word in all_labelII_words.split()])

label_II_wordcloud = WordCloud(background_color = 'black', width = 3000, height = 2500).generate(joined_labelII_words)

plt.figure(1, figsize = (12, 12))
plt.imshow(label_II_wordcloud)
plt.axis('off')
plt.show()

# Creating a wordcloud of cleaned descriptions labels with Potential Accident Level III

label_III = data[data['Potential Accident Level'] == 2]
all_labelIII_words = ' '.join(label_III['Cleaned Description'])
joined_labelIII_words = " ".join([word for word in all_labelIII_words.split()])

label_III_wordcloud = WordCloud(background_color = 'black', width = 3000, height = 2500).generate(joined_labelIII_words)

plt.figure(1, figsize = (12, 12))
plt.imshow(label_III_wordcloud)
plt.axis('off')
plt.show()

# Creating a wordcloud of cleaned descriptions labels with Potential Accident Level IV

label_IV = data[data['Potential Accident Level'] == 3]
all_labelIV_words = ' '.join(label_IV['Cleaned Description'])
joined_labelIV_words = " ".join([word for word in all_labelIV_words.split()])

label_IV_wordcloud = WordCloud(background_color = 'black', width = 3000, height = 2500).generate(joined_labelIV_words)

plt.figure(1, figsize = (12, 12))
plt.imshow(label_IV_wordcloud)
plt.axis('off')
plt.show()

# Creating a wordcloud of cleaned descriptions labels with Potential Accident Level V

label_V = data[data['Potential Accident Level'] == 4]
all_labelV_words = ' '.join(label_V['Cleaned Description'])
joined_labelV_words = " ".join([word for word in all_labelV_words.split()])

label_V_wordcloud = WordCloud(background_color = 'black', width = 3000, height = 2500).generate(joined_labelV_words)

plt.figure(1, figsize = (12, 12))
plt.imshow(label_V_wordcloud)
plt.axis('off')
plt.show()

PREPARING THE DATASETS

1. We will use the 'data' DataFrame as our base dataset for creating datasets for machine learning

2. First, we will use CountVectorizer, TF-IDF and Word2Vec to create numerical vectors of the 'Cleaned Description' column

3. Next, we will create the following datasets

• CountVectorized Cleaned Description, Potential Accident Level
• TF-IDF Cleaned Description, Potential Accident Level
• Word2Vec Cleaned Description, Potential Accident Level

1. We will also create the following datasets

• CountVectorized Cleaned Description, all other One-Hot Encoded dummy variables, Potential Accident Level
• TF-IDF Cleaned Description, all other One-Hot Encoded dummy variables, Potential Accident Level
• Word2Vec Cleaned Description, all other One-Hot Encoded dummy variables, Potential Accident Level

1. For each of the datasets above, we will also use SMOTE to oversample and create datasets with balanced target variables, after splitting each of these datasets into their training and test sets

Preparing Tokenizer Dataset

# Applying CountVectorizer & Processing Cleaned Description from 'data'

# We restrict this dataset to a maximum of 200 features, and apply an ngram_range of 1 to 3, so that contextual chunks are also tokenized

max_features = 200
maxlen=200

## Defining countvectorizer
tk = Tokenizer(num_words=max_features, oov_token='<OOV>')
tk.fit_on_texts(data['Cleaned Description'])
tk_corpus = tk.texts_to_sequences(data['Cleaned Description'])
tk_corpus = pad_sequences(tk_corpus, maxlen=maxlen)
print(tk_corpus)


# Creating the dataset that holds only the CountVectorized Cleaned Descriptions and the Potential Accident Level target variable
tk_df = pd.DataFrame(data = tk_corpus)
tk_df['Potential Accident Level'] = data['Potential Accident Level']

# Creating the full set - CountVectorized descriptions, other categorical columns which have been One-Hot Encoded and the Potential Accident Level target variable
tk_fullset = pd.concat([data, tk_df], axis = 1)

## Deleting Cleaned Description from the cv_fullset dataframe
cols_to_delete = ['Potential Accident Level', 'Cleaned Description']

# Final CountVectoried Full Set
tk_fullset = tk_fullset.drop(columns = cols_to_delete, axis = 1)
tk_fullset['Potential Accident Level'] = data['Potential Accident Level']

[[ 0  0  0 ... 92  1  1]
 [ 0  0  0 ...  1  1  1]
 [ 0  0  0 ... 15  5 11]
 ...
 [ 0  0  0 ... 16  6  3]
 [ 0  0  0 ... 16  6  3]
 [ 0  0  0 ...  1 41 11]]

tk_df.head(5)

	0	1	2	3	4	5	6	7	8	9	...	191	192	193	194	195	196	197	198	199	Potential Accident Level
0	0	0	0	0	0	0	0	0	0	0	...	27	1	16	56	36	87	92	1	1	3
1	0	0	0	0	0	0	0	0	0	0	...	1	1	1	1	1	1	1	1	1	3
2	0	0	0	0	0	0	0	0	0	0	...	33	1	1	17	6	57	15	5	11	2
3	0	0	0	0	0	0	0	0	0	0	...	1	93	78	77	1	6	3	5	11	0
4	0	0	0	0	0	0	0	0	0	0	...	1	23	1	7	57	14	5	11	41	3

5 rows × 201 columns

tk_fullset.head(5)

	Countries_Country_02	Countries_Country_03	Local_Local_02	Local_Local_03	Local_Local_04	Local_Local_05	Local_Local_06	Local_Local_07	Local_Local_08	Local_Local_09	...	191	192	193	194	195	196	197	198	199	Potential Accident Level
0	0	0	0	0	0	0	0	0	0	0	...	27	1	16	56	36	87	92	1	1	3
1	1	0	1	0	0	0	0	0	0	0	...	1	1	1	1	1	1	1	1	1	3
2	0	0	0	1	0	0	0	0	0	0	...	33	1	1	17	6	57	15	5	11	2
3	0	0	0	0	1	0	0	0	0	0	...	1	93	78	77	1	6	3	5	11	0
4	0	0	0	0	1	0	0	0	0	0	...	1	23	1	7	57	14	5	11	41	3

5 rows × 220 columns

Preparing CountVectorized Dataset

# Applying CountVectorizer & Processing Cleaned Description from 'data'

# We restrict this dataset to a maximum of 200 features, and apply an ngram_range of 1 to 3, so that contextual chunks are also tokenized

max_features = 200

## Defining countvectorizer
cv = CountVectorizer(max_features = max_features, ngram_range = (1, 3))
cv_corpus = cv.fit_transform(data['Cleaned Description'])

# Creating the dataset that holds only the CountVectorized Cleaned Descriptions and the Potential Accident Level target variable
cv_df = pd.DataFrame(data = cv_corpus.toarray(), columns = cv.get_feature_names_out())
cv_df['Potential Accident Level'] = data['Potential Accident Level']


# Creating the full set - CountVectorized descriptions, other categorical columns which have been One-Hot Encoded and the Potential Accident Level target variable
cv_fullset = pd.concat([data, cv_df], axis = 1)

## Deleting Cleaned Description from the cv_fullset dataframe
cols_to_delete = ['Potential Accident Level', 'Cleaned Description']

# Final CountVectoried Full Set
cv_fullset = cv_fullset.drop(columns = cols_to_delete, axis = 1)
cv_fullset['Potential Accident Level'] = data['Potential Accident Level']

cv_df.head()

	access	accident	acid	activity	air	another	approx	approximately	area	arm	...	wearing	weight	when	while	without	work	worker	workshop	wound	Potential Accident Level
0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	1	0	0	0	0	0	3
1	0	0	0	0	0	0	0	0	1	0	...	0	0	0	0	0	0	0	0	0	3
2	0	0	0	0	0	0	0	0	1	0	...	0	0	1	0	0	1	0	0	0	2
3	0	0	0	0	0	0	0	1	0	0	...	0	0	1	0	0	0	0	0	0	0
4	0	0	0	0	0	0	0	1	0	0	...	0	1	0	0	0	0	1	0	0	3

5 rows × 201 columns

cv_fullset.head()

	Countries_Country_02	Countries_Country_03	Local_Local_02	Local_Local_03	Local_Local_04	Local_Local_05	Local_Local_06	Local_Local_07	Local_Local_08	Local_Local_09	...	wearing	weight	when	while	without	work	worker	workshop	wound	Potential Accident Level
0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	1	0	0	0	0	0	3
1	1	0	1	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	3
2	0	0	0	1	0	0	0	0	0	0	...	0	0	1	0	0	1	0	0	0	2
3	0	0	0	0	1	0	0	0	0	0	...	0	0	1	0	0	0	0	0	0	0
4	0	0	0	0	1	0	0	0	0	0	...	0	1	0	0	0	0	1	0	0	3

5 rows × 220 columns

Preparing the TF-IDF vectorized Set

# Applying TfidfVectorizer & Processing Cleaned Description from 'data'

# We restrict this dataset to a maximum of 200 features, and apply an ngram_range of 1 to 3, so that contextual chunks are also tokenized

max_features = 200

## Defining TfidfVectorizer
tfidf = TfidfVectorizer(max_features = max_features, ngram_range = (1, 3))
tfidf_corpus = tfidf.fit_transform(data['Cleaned Description'])

# Creating the dataset that holds only the TfidfVectorizer Cleaned Descriptions and the Potential Accident Level target variable
tfidf_df = pd.DataFrame(data = tfidf_corpus.toarray(), columns = tfidf.get_feature_names_out())
tfidf_df['Potential Accident Level'] = data['Potential Accident Level']

# Creating the full set - TfidfVectorizer descriptions, other categorical columns which have been One-Hot Encoded and the Potential Accident Level target variable
tfidf_fullset = pd.concat([data, tfidf_df], axis = 1)

## Deleting Cleaned Description from the cv_fullset dataframe
cols_to_delete = ['Potential Accident Level', 'Cleaned Description']

# Final TfidfVectorizer Full Set
tfidf_fullset = tfidf_fullset.drop(columns = cols_to_delete, axis = 1)
tfidf_fullset['Potential Accident Level'] = data['Potential Accident Level']

tfidf_fullset.head()

	Countries_Country_02	Countries_Country_03	Local_Local_02	Local_Local_03	Local_Local_04	Local_Local_05	Local_Local_06	Local_Local_07	Local_Local_08	Local_Local_09	...	wearing	weight	when	while	without	work	worker	workshop	wound	Potential Accident Level
0	0	0	0	0	0	0	0	0	0	0	...	0.0	0.000000	0.000000	0.148382	0.0	0.000000	0.000000	0.0	0.0	3
1	1	0	1	0	0	0	0	0	0	0	...	0.0	0.000000	0.000000	0.000000	0.0	0.000000	0.000000	0.0	0.0	3
2	0	0	0	1	0	0	0	0	0	0	...	0.0	0.000000	0.106321	0.000000	0.0	0.177103	0.000000	0.0	0.0	2
3	0	0	0	0	1	0	0	0	0	0	...	0.0	0.000000	0.070892	0.000000	0.0	0.000000	0.000000	0.0	0.0	0
4	0	0	0	0	1	0	0	0	0	0	...	0.0	0.236551	0.000000	0.000000	0.0	0.000000	0.182236	0.0	0.0	3

5 rows × 220 columns

tfidf_df.head()

	access	accident	acid	activity	air	another	approx	approximately	area	arm	...	wearing	weight	when	while	without	work	worker	workshop	wound	Potential Accident Level
0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.000000	0.000000	0.0	...	0.0	0.000000	0.000000	0.148382	0.0	0.000000	0.000000	0.0	0.0	3
1	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.000000	0.339654	0.0	...	0.0	0.000000	0.000000	0.000000	0.0	0.000000	0.000000	0.0	0.0	3
2	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.000000	0.185894	0.0	...	0.0	0.000000	0.106321	0.000000	0.0	0.177103	0.000000	0.0	0.0	2
3	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.135054	0.000000	0.0	...	0.0	0.000000	0.070892	0.000000	0.0	0.000000	0.000000	0.0	0.0	0
4	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.195180	0.000000	0.0	...	0.0	0.236551	0.000000	0.000000	0.0	0.000000	0.182236	0.0	0.0	3

5 rows × 201 columns

Preparing the Word2Vec dataset

# Tokenizing the sentences in the Cleaned Description column (using NLTK's tokenize)
tokenized_sentences = [word_tokenize(sentence) for sentence in data['Cleaned Description']]

# Training the Word2Vec model
word2vec_model = Word2Vec(sentences = tokenized_sentences,
                          vector_size = 200,
                          window = 5,
                          min_count = 1,
                          workers = 4)

# Using the same tokenized sentences for word2vec
word2vec_descriptions = np.array([np.mean([word2vec_model.wv[word] for word in sentence], axis = 0) for sentence in tokenized_sentences])

# Creating column names for the dataframe
word2vec_columns = [f'word2vec_{i}' for i in range(word2vec_descriptions.shape[1])]

# Creating the dataframe
wv_fullset = pd.concat([data.drop('Cleaned Description', axis = 1), pd.DataFrame(word2vec_descriptions, columns = word2vec_columns)], axis = 1)

# Cols to drop to get only the word2vec vectors, and Potential Accident Level, and drop all the dummy variables
cols_to_drop = ['Countries_Country_02', 'Countries_Country_03', 'Local_Local_02', 'Local_Local_03', 'Local_Local_04', 'Local_Local_05', 'Local_Local_06',
                'Local_Local_07', 'Local_Local_08', 'Local_Local_09', 'Local_Local_10', 'Local_Local_11', 'Local_Local_12',
                'Industry Sector_Mining', 'Industry Sector_Others',
                'Gender_Male',
                'Personnel Type_Third Party', 'Personnel Type_Third Party (Remote)',
                'Critical Risk_Uncategorized']

wv_df = wv_fullset.drop(columns = cols_to_drop, axis = 1)

wv_df.head()

	Potential Accident Level	word2vec_0	word2vec_1	word2vec_2	word2vec_3	word2vec_4	word2vec_5	word2vec_6	word2vec_7	word2vec_8	...	word2vec_190	word2vec_191	word2vec_192	word2vec_193	word2vec_194	word2vec_195	word2vec_196	word2vec_197	word2vec_198	word2vec_199
0	3	0.004326	-0.001519	0.000794	0.008592	0.011592	-0.009999	-0.000350	0.023833	-0.005423	...	0.007718	-0.007817	-0.005565	-0.010294	0.010797	0.005050	0.006894	-0.012566	0.000532	-0.007252
1	3	0.001403	0.000525	-0.000086	0.005281	0.004167	-0.003324	-0.000772	0.010607	-0.003369	...	0.003500	-0.003544	-0.001804	-0.002975	0.003941	0.001357	0.002372	-0.004865	-0.000097	-0.001936
2	2	0.005551	-0.000299	0.000724	0.009850	0.014585	-0.011649	0.001294	0.025517	-0.007976	...	0.008945	-0.007965	-0.007316	-0.009868	0.011882	0.004525	0.007796	-0.015325	0.000420	-0.007520
3	0	0.003297	-0.000740	0.000201	0.007622	0.010473	-0.008884	0.001020	0.020591	-0.005865	...	0.006896	-0.007172	-0.005034	-0.008812	0.009179	0.004475	0.006574	-0.011668	-0.000057	-0.006490
4	3	0.003706	-0.000886	0.001187	0.006009	0.009844	-0.009281	0.000368	0.018744	-0.004941	...	0.007438	-0.005714	-0.005770	-0.007221	0.008052	0.003302	0.005709	-0.010173	0.000193	-0.005363

5 rows × 201 columns

wv_fullset.head()

	Potential Accident Level	Countries_Country_02	Countries_Country_03	Local_Local_02	Local_Local_03	Local_Local_04	Local_Local_05	Local_Local_06	Local_Local_07	Local_Local_08	...	word2vec_190	word2vec_191	word2vec_192	word2vec_193	word2vec_194	word2vec_195	word2vec_196	word2vec_197	word2vec_198	word2vec_199
0	3	0	0	0	0	0	0	0	0	0	...	0.007718	-0.007817	-0.005565	-0.010294	0.010797	0.005050	0.006894	-0.012566	0.000532	-0.007252
1	3	1	0	1	0	0	0	0	0	0	...	0.003500	-0.003544	-0.001804	-0.002975	0.003941	0.001357	0.002372	-0.004865	-0.000097	-0.001936
2	2	0	0	0	1	0	0	0	0	0	...	0.008945	-0.007965	-0.007316	-0.009868	0.011882	0.004525	0.007796	-0.015325	0.000420	-0.007520
3	0	0	0	0	0	1	0	0	0	0	...	0.006896	-0.007172	-0.005034	-0.008812	0.009179	0.004475	0.006574	-0.011668	-0.000057	-0.006490
4	3	0	0	0	0	1	0	0	0	0	...	0.007438	-0.005714	-0.005770	-0.007221	0.008052	0.003302	0.005709	-0.010173	0.000193	-0.005363

5 rows × 220 columns

Checking the shape of the datasets created above

# CountVectorized truncated set

cv_df.shape

(411, 201)

# CountVectorized full set

cv_fullset.shape

(411, 220)

# TFIDF truncated set

tfidf_df.shape

(411, 201)

# TFIDF full set

tfidf_fullset.shape

(411, 220)

# Word2Vec truncated set

wv_df.shape

(411, 201)

# Word2Vec full set

wv_fullset.shape

(411, 220)

tk_df.shape

(411, 201)

tk_fullset.shape

(411, 220)

Splitting the datasets into X & y

# Splitting the datasets into X & y

# CountVectorizer Sets

# Truncated Dataset
X_cv_df = cv_df.drop('Potential Accident Level', axis = 1)
y_cv_df = cv_df['Potential Accident Level']

# Full Dataset
X_cv_fullset = cv_fullset.drop('Potential Accident Level', axis = 1)
y_cv_fullset = cv_fullset['Potential Accident Level']

# TFIDF Sets

# Truncated Dataset
X_tfidf_df = tfidf_df.drop('Potential Accident Level', axis = 1)
y_tfidf_df = tfidf_df['Potential Accident Level']

# Full Dataset
X_tfidf_fullset = tfidf_fullset.drop('Potential Accident Level', axis = 1)
y_tfidf_fullset = tfidf_fullset['Potential Accident Level']

# Word2Vec Sets

# Truncated Dataset
X_wv_df = wv_df.drop('Potential Accident Level', axis = 1)
y_wv_df = wv_df['Potential Accident Level']

# Full Dataset
X_wv_fullset = wv_fullset.drop('Potential Accident Level', axis = 1)
y_wv_fullset = wv_fullset['Potential Accident Level']

# Tokenizer Sets

# Truncated Dataset
X_tk_df = tk_df.drop('Potential Accident Level', axis = 1)
y_tk_df = tk_df['Potential Accident Level']

# Full Dataset
X_tk_fullset = tk_fullset.drop('Potential Accident Level', axis = 1)
y_tk_fullset = tk_fullset['Potential Accident Level']

Splitting the X's & y's into train and test sets

# CountVectorized truncated set
X_train_cv, X_test_cv, y_train_cv, y_test_cv = train_test_split(X_cv_df, y_cv_df, test_size = 0.2, stratify = y_cv_df, random_state = seed)

# CountVectorized full set
X_train_cvfull, X_test_cvfull, y_train_cvfull, y_test_cvfull = train_test_split(X_cv_fullset, y_cv_fullset, test_size = 0.2, stratify = y_cv_fullset, random_state = seed)

# TFIDF truncated set
X_train_tfidf, X_test_tfidf, y_train_tfidf, y_test_tfidf = train_test_split(X_tfidf_df, y_tfidf_df, test_size = 0.2, stratify = y_tfidf_df, random_state = seed)

# TFIDF full set
X_train_tfidffull, X_test_tfidffull, y_train_tfidffull, y_test_tfidffull = train_test_split(X_tfidf_fullset, y_tfidf_fullset, test_size = 0.2, stratify = y_tfidf_fullset, random_state = seed)

# Word2Vec truncated set
X_train_wv, X_test_wv, y_train_wv, y_test_wv = train_test_split(X_wv_df, y_wv_df, test_size = 0.2, stratify = y_wv_df, random_state = seed)

# Word2Vec full set
X_train_wvfull, X_test_wvfull, y_train_wvfull, y_test_wvfull = train_test_split(X_wv_fullset, y_wv_fullset, test_size = 0.2, stratify = y_wv_fullset, random_state = seed)

# Word2Vec truncated set
X_train_tk, X_test_tk, y_train_tk, y_test_tk = train_test_split(X_tk_df, y_tk_df, test_size = 0.2, stratify = y_tk_df, random_state = seed)

# Word2Vec full set
X_train_tkfull, X_test_tkfull, y_train_tkfull, y_test_tkfull = train_test_split(X_tk_fullset, y_tk_fullset, test_size = 0.2, stratify = y_tk_fullset, random_state = seed)

# Creating oversampled sets using SMOTE

# Truncated CountVectorized Set
sm1 = SMOTE(random_state = seed)
X_train_cv_smote, y_train_cv_smote = sm1.fit_resample(X_train_cv,y_train_cv)

# Full CountVectorized Set
sm2 = SMOTE(random_state = seed)
X_train_cvfull_smote, y_train_cvfull_smote = sm2.fit_resample(X_train_cvfull, y_train_cvfull)

# Truncated TFIDF Set
sm3 = SMOTE(random_state = seed)
X_train_tfidf_smote, y_train_tfidf_smote = sm3.fit_resample(X_train_tfidf, y_train_tfidf)

# Full TFIDF Set
sm4 = SMOTE(random_state = seed)
X_train_tfidffull_smote, y_train_tfidffull_smote = sm4.fit_resample(X_train_tfidffull, y_train_tfidffull)

# Truncated Word2Vec Set
sm5 = SMOTE(random_state = seed)
X_train_wv_smote, y_train_wv_smote = sm5.fit_resample(X_train_wv, y_train_wv)

# Full Word2Vec Set
sm6 = SMOTE(random_state = seed)
X_train_wvfull_smote, y_train_wvfull_smote = sm6.fit_resample(X_train_wvfull, y_train_wvfull)

# Truncated Tokenizer Set
sm7 = SMOTE(random_state = seed)
X_train_tk_smote, y_train_tk_smote = sm7.fit_resample(X_train_tk.values, y_train_tk)

# Full Word2Vec Set
sm8 = SMOTE(random_state = seed)
X_train_tkfull_smote, y_train_tkfull_smote = sm8.fit_resample(X_train_tkfull.values, y_train_tkfull)

MACHINE LEARNING

1. Truncated Datasets

# CountVectorizer - X_train_cv, X_test_cv, y_train_cv, y_test_cv
# TFIDF Vectorizer - X_train_tfidf, X_test_tfidf, y_train_tfidf, y_test_tfidf
# Word2Vec Vectorizer - X_train_wv, X_test_wv, y_train_wv, y_test_wv

1. Basic SL models on Truncated CountVectorized Set

# Truncated CountVectorized Set

truncated_cv_basic = ML_Models(X_train_cv, X_test_cv, y_train_cv, y_test_cv)

truncated_cv_basic

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.945122	0.421687
1	Multinomial NB	0.689024	0.421687
2	KNearestNeighbors	0.512195	0.265060
3	DecisionTreeClassifier	0.381098	0.265060
4	RandomForestClassifier	0.814024	0.301205
5	AdaBoostClassifier	0.442073	0.349398
6	GradientBoostClassifier	0.960366	0.337349
7	XGBoostClassifier	0.954268	0.385542

2. Basic SL models on Truncated CountVectorized SMOTE Set

# Truncated CountVectorized Set - SMOTE

truncated_cv_smote = ML_Models(X_train_cv_smote, X_test_cv, y_train_cv_smote, y_test_cv)

truncated_cv_smote

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.921818	0.337349
1	Multinomial NB	0.732727	0.361446
2	KNearestNeighbors	0.610909	0.168675
3	DecisionTreeClassifier	0.481818	0.216867
4	RandomForestClassifier	0.752727	0.313253
5	AdaBoostClassifier	0.463636	0.253012
6	GradientBoostClassifier	0.940000	0.361446
7	XGBoostClassifier	0.934545	0.313253

3. Basic SL models on Truncated TFIDF Set

# Truncated TFIDF Set

truncated_tfidf_basic = ML_Models(X_train_tfidf, X_test_tfidf, y_train_tfidf, y_test_tfidf)

truncated_tfidf_basic

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.719512	0.433735
1	Multinomial NB	0.625000	0.397590
2	KNearestNeighbors	0.551829	0.349398
3	DecisionTreeClassifier	0.463415	0.289157
4	RandomForestClassifier	0.871951	0.349398
5	AdaBoostClassifier	0.393293	0.313253
6	GradientBoostClassifier	0.996951	0.349398
7	XGBoostClassifier	0.996951	0.349398

4. Basic SL models on Truncated TFIDF - SMOTE Set

# Truncated TFIDF Set - SMOTE

truncated_tfidf_smote = ML_Models(X_train_tfidf_smote, X_test_tfidf, y_train_tfidf_smote, y_test_tfidf)

truncated_tfidf_smote

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.852727	0.349398
1	Multinomial NB	0.787273	0.337349
2	KNearestNeighbors	0.670909	0.253012
3	DecisionTreeClassifier	0.576364	0.313253
4	RandomForestClassifier	0.885455	0.325301
5	AdaBoostClassifier	0.434545	0.216867
6	GradientBoostClassifier	1.000000	0.397590
7	XGBoostClassifier	0.998182	0.361446

5. Basic SL models on Truncated Word2Vec Set

# Truncated Word2Vec Set

truncated_wv_basic = ML_Models_without_MNB(X_train_wv, X_test_wv, y_train_wv, y_test_wv)

truncated_wv_basic

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.335366	0.337349
1	KNearestNeighbors	0.509146	0.192771
2	DecisionTreeClassifier	0.621951	0.180723
3	RandomForestClassifier	0.966463	0.289157
4	AdaBoostClassifier	0.423780	0.253012
5	GradientBoostClassifier	1.000000	0.385542
6	XGBoostClassifier	1.000000	0.433735

6. Basic SL models on Truncated Word2Vec SMOTE Set

# Truncated Word2Vec Set - SMOTE

truncated_wv_smote = ML_Models_without_MNB(X_train_wv_smote, X_test_wv, y_train_wv_smote, y_test_wv)

truncated_wv_smote

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.272727	0.240964
1	KNearestNeighbors	0.665455	0.072289
2	DecisionTreeClassifier	0.800000	0.313253
3	RandomForestClassifier	0.952727	0.301205
4	AdaBoostClassifier	0.563636	0.168675
5	GradientBoostClassifier	1.000000	0.313253
6	XGBoostClassifier	1.000000	0.409639

OBSERVATIONS

1. CountVectorized Data

• We note that the basic machine learning models do quite poorly on this dataset
• All models exhibit significant amounts of overfitting
• With this context, the Multinomial Naive Bayes model and Logistic Regression models have relatively higher accuracy levels.
• However, from the confusion matrices of these models, we note that all models do a bad job of predicting actual classes on their respective test sets
• Applying SMOTE does not improve the performance of these models, either on accuracy, or on the models abilities to correctly predict the right classes in the test sets. We also see a significant amount of overfitting in the SMOTE models as well

1. TFIDF Data

• We note that the basic TFIDF vectorized models return accuracies at par with or poorer than their CountVectorized counterparts.
• As was the case with the CountVectorized datasets, the models perform equally poorly on classifying data according to their actual classes.
• While the SMOTE models do appear to perform trivially better on their accuracy scores in the test sets as compared to the non-oversampled TFIDF data, they are still lower than the accuracy levels seen on the CountVectorized basic models.
• The SMOTE models also perform poorly on accurately classifying the data according to their actual classes

1. Word2Vec

• In the Word2Vec vectorized models, we do not consider the Multinomial Naive Bayes model, since it does not take in negative values prevalent in the vectorized embeddings
• Similar to the other datasets, we see that the basic ML models perform poorly in terms of accuracy, compared to both the TFIDF datasets and the CountVectorized Datasets.
• The accuracy results using SMOTE drop accuracy scores even further.
• We note that, like in the case of all other models described above, both the basic Word2Vec models and the SMOTE models do a very poor job of actually classifying observations according to their actual labels.

Accordingly, we will now tune these models to see if we can achieve better results. Given that the SMOTE models were unable to improve model performance in any of the datasets, we will only fine-tune the basic models going forward

Running GridSearchCV using StratifiedKFold cross validation in an attempt to improve model performance

# On Truncated CountVectorized Set

ML_Tuned_Models(X_train_cv, X_test_cv, y_train_cv, y_test_cv)

 model: RanForCls
 best parameters: 
{'ccp_alpha': 0.001, 'criterion': 'gini', 'max_depth': 12, 'max_features': 'sqrt'}
 model: AdaBoosCls
 best parameters: 
{'learning_rate': 1, 'n_estimators': 100}
 model: LogReg
 best parameters: 
{'C': 1, 'penalty': 'l2'}
 model: KNN
 best parameters: 
{'metric': 'euclidean', 'n_neighbors': 3, 'weights': 'distance'}
 model: Multinomial NB
 best parameters: 
{'alpha': 0.5, 'fit_prior': True}

	model	train best score	test best score
0	RanForCls	0.935976	0.397590
1	AdaBoosCls	0.460366	0.301205
2	LogReg	0.945122	0.421687
3	KNN	1.000000	0.313253
4	Multinomial NB	0.689024	0.421687

# On Truncated TFIDF Set

ML_Tuned_Models(X_train_tfidf, X_test_tfidf, y_train_tfidf, y_test_tfidf)

 model: RanForCls
 best parameters: 
{'ccp_alpha': 0.01, 'criterion': 'entropy', 'max_depth': 11, 'max_features': 'sqrt'}
 model: AdaBoosCls
 best parameters: 
{'learning_rate': 0.05, 'n_estimators': 500}
 model: LogReg
 best parameters: 
{'C': 100, 'penalty': 'l2'}
 model: KNN
 best parameters: 
{'metric': 'euclidean', 'n_neighbors': 11, 'weights': 'distance'}
 model: Multinomial NB
 best parameters: 
{'alpha': 0.5, 'fit_prior': True}

	model	train best score	test best score
0	RanForCls	0.923780	0.361446
1	AdaBoosCls	0.457317	0.385542
2	LogReg	1.000000	0.373494
3	KNN	1.000000	0.433735
4	Multinomial NB	0.655488	0.397590

# On Truncated Word2Vec Set

ML_Tuned_Models_without_MNB(X_train_wv, X_test_wv, y_train_wv, y_test_wv)

 model: RanForCls
 best parameters: 
{'ccp_alpha': 0.01, 'criterion': 'gini', 'max_depth': 10, 'max_features': 'sqrt'}
 model: AdaBoosCls
 best parameters: 
{'learning_rate': 0.05, 'n_estimators': 50}
 model: LogReg
 best parameters: 
{'C': 100, 'penalty': 'l2'}
 model: KNN
 best parameters: 
{'metric': 'euclidean', 'n_neighbors': 19, 'weights': 'distance'}

	model	train best score	test best score
0	RanForCls	0.960366	0.301205
1	AdaBoosCls	0.356707	0.337349
2	LogReg	0.362805	0.349398
3	KNN	1.000000	0.277108

# Defining a function to train data on the tuned CountVectorized dataset

def ML_Models_CV(X_train, X_test, y_train, y_test):

  models={
      "LogisticRegression":LogisticRegression(multi_class='multinomial', C = 1, penalty = 'l2', solver='saga', max_iter=10000,random_state = seed),
      "Multinomial NB": MultinomialNB(alpha = 0.5, fit_prior = True),
      "KNearestNeighbors": KNeighborsClassifier(metric = 'euclidean', n_neighbors = 3, weights = 'distance'),
      "DecisionTreeClassifier":DecisionTreeClassifier(criterion='entropy',class_weight = 'balanced', max_depth = 6,random_state = seed, min_samples_leaf = 5),
      "RandomForestClassifier":RandomForestClassifier(ccp_alpha = 0.001, class_weight = 'balanced',criterion = 'gini', n_estimators=100, max_features = 'sqrt', max_depth = 12,random_state = seed),
      "AdaBoostClassifier":AdaBoostClassifier(random_state = seed, learning_rate = 1, n_estimators = 100),
      "GradientBoostClassifier":GradientBoostingClassifier(random_state = seed),
      'XGBoostClassifier': xgb.XGBClassifier(colsample_bytree= 0.6, gamma= 0.5, max_depth= 5, min_child_weight= 1, subsample= 0.8,n_estimators=100)
  }


  names = []
  train_scores = []
  test_scores = []

  for name, model in models.items():
    model.fit(X_train, y_train)
    result_train = accuracy_score(y_train, model.predict(X_train))
    result_test = accuracy_score(y_test, model.predict(X_test))
    names.append(name)
    train_scores.append(result_train) # Appending the test scores to the list
    test_scores.append(result_test) # Appending the test scores to the list
    result_df = pd.DataFrame({'model': names, 'Train accuracy': train_scores, 'Test accuracy': test_scores})

    cm = confusion_matrix(y_test, model.predict(X_test))

    fig, ax = plt.subplots(nrows = 1, ncols = 1, figsize = (5,4))
    sns.heatmap(cm,
                annot=True,
                fmt='g',
                xticklabels=['I','II','III','IV','V'],
                yticklabels=['I','II','III','IV','V'], ax = ax)
    ax.set_ylabel('Prediction', fontsize = 11)
    ax.set_xlabel('Actual',fontsize = 11)
    ax.set_title(f'Confusion Matrix: {name} model',fontsize = 14)
    plt.show()

  return result_df

# Defining a function to train data on the tuned TFIDF dataset

def ML_Models_tfidf(X_train, X_test, y_train, y_test):

  models={
      "LogisticRegression":LogisticRegression(multi_class='multinomial', C = 100, penalty = 'l2', solver='saga', max_iter=10000,random_state = seed),
      "Multinomial NB": MultinomialNB(alpha = 0.5, fit_prior = True),
      "KNearestNeighbors": KNeighborsClassifier(metric = 'euclidean', n_neighbors = 11, weights = 'distance'),
      "DecisionTreeClassifier":DecisionTreeClassifier(criterion='entropy',class_weight = 'balanced', max_depth = 6,random_state = seed, min_samples_leaf = 5),
      "RandomForestClassifier":RandomForestClassifier(ccp_alpha = 0.01, class_weight = 'balanced',criterion = 'entropy', n_estimators=100, max_features = 'sqrt', max_depth = 11,random_state = seed),
      "AdaBoostClassifier":AdaBoostClassifier(random_state = seed, learning_rate = 0.05, n_estimators = 500),
      "GradientBoostClassifier":GradientBoostingClassifier(random_state = seed),
      'XGBoostClassifier': xgb.XGBClassifier(colsample_bytree= 0.6, gamma= 0.5, max_depth= 5, min_child_weight= 1, subsample= 0.8,n_estimators=100)
  }


  names = []
  train_scores = []
  test_scores = []

  for name, model in models.items():
    model.fit(X_train, y_train)
    result_train = accuracy_score(y_train, model.predict(X_train))
    result_test = accuracy_score(y_test, model.predict(X_test))
    names.append(name)
    train_scores.append(result_train) # Appending the test scores to the list
    test_scores.append(result_test) # Appending the test scores to the list
    result_df = pd.DataFrame({'model': names, 'Train accuracy': train_scores, 'Test accuracy': test_scores})

    cm = confusion_matrix(y_test, model.predict(X_test))

    fig, ax = plt.subplots(nrows = 1, ncols = 1, figsize = (5,4))
    sns.heatmap(cm,
                annot=True,
                fmt='g',
                xticklabels=['I','II','III','IV','V'],
                yticklabels=['I','II','III','IV','V'], ax = ax)
    ax.set_ylabel('Prediction', fontsize = 11)
    ax.set_xlabel('Actual',fontsize = 11)
    ax.set_title(f'Confusion Matrix: {name} model',fontsize = 14)
    plt.show()

  return result_df

# Defining a function to train data on the tuned Word2Vec dataset

def ML_Models_wv(X_train, X_test, y_train, y_test):

  models={
      "LogisticRegression":LogisticRegression(multi_class='multinomial', C = 100, penalty = 'l2', solver='saga', max_iter=10000,random_state = seed),
    #   "Multinomial NB": MultinomialNB(alpha = 0.5, fit_prior = True),
      "KNearestNeighbors": KNeighborsClassifier(metric = 'euclidean', n_neighbors = 19, weights = 'distance'),
      "DecisionTreeClassifier":DecisionTreeClassifier(criterion='entropy',class_weight = 'balanced', max_depth = 6,random_state = seed, min_samples_leaf = 5),
      "RandomForestClassifier":RandomForestClassifier(ccp_alpha = 0.01, class_weight = 'balanced',criterion = 'gini', n_estimators=100, max_features = 'sqrt', max_depth = 10,random_state = seed),
      "AdaBoostClassifier":AdaBoostClassifier(random_state = seed, learning_rate = 0.05, n_estimators = 50),
      "GradientBoostClassifier":GradientBoostingClassifier(random_state = seed),
      'XGBoostClassifier': xgb.XGBClassifier(colsample_bytree= 0.6, gamma= 0.5, max_depth= 5, min_child_weight= 1, subsample= 0.8,n_estimators=100)
  }


  names = []
  train_scores = []
  test_scores = []

  for name, model in models.items():
    model.fit(X_train, y_train)
    result_train = accuracy_score(y_train, model.predict(X_train))
    result_test = accuracy_score(y_test, model.predict(X_test))
    names.append(name)
    train_scores.append(result_train) # Appending the test scores to the list
    test_scores.append(result_test) # Appending the test scores to the list
    result_df = pd.DataFrame({'model': names, 'Train accuracy': train_scores, 'Test accuracy': test_scores})

    cm = confusion_matrix(y_test, model.predict(X_test))

    fig, ax = plt.subplots(nrows = 1, ncols = 1, figsize = (5,4))
    sns.heatmap(cm,
                annot=True,
                fmt='g',
                xticklabels=['I','II','III','IV','V'],
                yticklabels=['I','II','III','IV','V'], ax = ax)
    ax.set_ylabel('Prediction', fontsize = 11)
    ax.set_xlabel('Actual',fontsize = 11)
    ax.set_title(f'Confusion Matrix: {name} model',fontsize = 14)
    plt.show()

  return result_df

# Training and testing tuned CountVectorized Datasets

ML_Models_CV(X_train_cv, X_test_cv, y_train_cv, y_test_cv)

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.945122	0.421687
1	Multinomial NB	0.689024	0.421687
2	KNearestNeighbors	1.000000	0.313253
3	DecisionTreeClassifier	0.381098	0.265060
4	RandomForestClassifier	0.954268	0.397590
5	AdaBoostClassifier	0.460366	0.301205
6	GradientBoostClassifier	0.960366	0.337349
7	XGBoostClassifier	0.954268	0.385542

# Training and testing tuned TFIDF Datasets

ML_Models_tfidf(X_train_tfidf, X_test_tfidf, y_train_tfidf, y_test_tfidf)

	model	Train accuracy	Test accuracy
0	LogisticRegression	1.000000	0.373494
1	Multinomial NB	0.655488	0.397590
2	KNearestNeighbors	1.000000	0.433735
3	DecisionTreeClassifier	0.463415	0.289157
4	RandomForestClassifier	0.963415	0.409639
5	AdaBoostClassifier	0.457317	0.385542
6	GradientBoostClassifier	0.996951	0.349398
7	XGBoostClassifier	0.996951	0.349398

# Training and testing tuned Word2Vec Datasets

ML_Models_wv(X_train_wv, X_test_wv, y_train_wv, y_test_wv)

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.362805	0.349398
1	KNearestNeighbors	1.000000	0.277108
2	DecisionTreeClassifier	0.621951	0.180723
3	RandomForestClassifier	0.972561	0.397590
4	AdaBoostClassifier	0.356707	0.337349
5	GradientBoostClassifier	1.000000	0.385542
6	XGBoostClassifier	1.000000	0.433735

Observations

1. From a comparative analysis of the accuracies and confusion matrices of the tuned datasets, we see that fine tuning the models did not significantly improve the base models.

2. It is clear that there is a significant amount of overfitting in the models.

3. None of the tuned models are able to effectively classify their actual class labels accurately to a great degree.

4. Thus, we are unable to clearly determine that fine tuning the base models led to any improvement in performance.

5. Given the poor accuracy scores of all the models above, model overfitting and the fact that their confusion matrices exhibit that the models are unable to accurately classify datapoints according to their actual labels, we are unable to clearly determine the 'best' model to close on, at this point.

6. The best 'relative' model we have seen so far is the basic Multinomial Naive Bayes model, with the caveat that it is still a weak model!

FULL DATASETS

# X_train_cvfull, X_test_cvfull, y_train_cvfull, y_test_cvfull
# X_train_tfidffull, X_test_tfidffull, y_train_tfidffull, y_test_tfidffull
# X_train_wvfull, X_test_wvfull, y_train_wvfull, y_test_wvfull
# X_train_cvfull_smote, y_train_cvfull_smote
# X_train_tfidffull_smote, y_train_tfidffull_smote
# X_train_wvfull_smote, y_train_wvfull_smote

# Full CountVectorized Set

full_cv_basic = ML_Models(X_train_cvfull, X_test_cvfull, y_train_cvfull, y_test_cvfull)

full_cv_basic

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.966463	0.421687
1	Multinomial NB	0.692073	0.385542
2	KNearestNeighbors	0.570122	0.277108
3	DecisionTreeClassifier	0.506098	0.373494
4	RandomForestClassifier	0.780488	0.457831
5	AdaBoostClassifier	0.399390	0.361446
6	GradientBoostClassifier	0.981707	0.313253
7	XGBoostClassifier	0.966463	0.433735

# Full CountVectorized Set - SMOTE

full_cvfull_smote = ML_Models(X_train_cvfull_smote, X_test_cvfull, y_train_cvfull_smote, y_test_cvfull)

full_cvfull_smote

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.952727	0.433735
1	Multinomial NB	0.760000	0.433735
2	KNearestNeighbors	0.645455	0.216867
3	DecisionTreeClassifier	0.603636	0.289157
4	RandomForestClassifier	0.770909	0.409639
5	AdaBoostClassifier	0.550909	0.265060
6	GradientBoostClassifier	0.965455	0.409639
7	XGBoostClassifier	0.960000	0.385542

# Full TFIDF Set

full_tfidf_basic = ML_Models(X_train_tfidffull, X_test_tfidffull, y_train_tfidffull, y_test_tfidffull)

full_tfidf_basic

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.725610	0.409639
1	Multinomial NB	0.560976	0.469880
2	KNearestNeighbors	0.594512	0.421687
3	DecisionTreeClassifier	0.472561	0.397590
4	RandomForestClassifier	0.844512	0.337349
5	AdaBoostClassifier	0.396341	0.325301
6	GradientBoostClassifier	1.000000	0.385542
7	XGBoostClassifier	1.000000	0.385542

# Full TFIDF Set - SMOTE

full_tfidf_smote = ML_Models(X_train_tfidffull_smote, X_test_tfidffull, y_train_tfidffull_smote, y_test_tfidffull)

full_tfidf_smote

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.874545	0.373494
1	Multinomial NB	0.696364	0.397590
2	KNearestNeighbors	0.720000	0.301205
3	DecisionTreeClassifier	0.598182	0.240964
4	RandomForestClassifier	0.880000	0.397590
5	AdaBoostClassifier	0.421818	0.240964
6	GradientBoostClassifier	0.998182	0.373494
7	XGBoostClassifier	0.996364	0.361446

# Full Word2Vec Set

full_wv_basic = ML_Models_without_MNB(X_train_wvfull, X_test_wvfull, y_train_wvfull, y_test_wvfull)

full_wv_basic

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.481707	0.421687
1	KNearestNeighbors	0.570122	0.301205
2	DecisionTreeClassifier	0.554878	0.216867
3	RandomForestClassifier	0.957317	0.421687
4	AdaBoostClassifier	0.359756	0.253012
5	GradientBoostClassifier	1.000000	0.361446
6	XGBoostClassifier	1.000000	0.373494

# Full Word2Vec Set - SMOTE

full_wv_smote = ML_Models_without_MNB(X_train_wvfull_smote, X_test_wvfull, y_train_wvfull_smote, y_test_wvfull)

full_wv_smote

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.532727	0.337349
1	KNearestNeighbors	0.681818	0.313253
2	DecisionTreeClassifier	0.701818	0.240964
3	RandomForestClassifier	0.918182	0.337349
4	AdaBoostClassifier	0.427273	0.180723
5	GradientBoostClassifier	1.000000	0.361446
6	XGBoostClassifier	1.000000	0.361446

OBSERVATIONS

1. We see that the performance of the models, for each of the full CountVectorized, TFIDF and Word2Vec vectorized datasets, has shown improvement compared to the truncated models.

2. However, the models do not show significant improvement after oversampling. Thus, we will proceed with the base line full datasets going forward.

3. As seen in the truncated models, the basic SL models also do not do a good job of predicting actual classes for the test set.

4. However, given their improved performance over the truncated datasets in general, it is clear that the full datasets are more amenable to predictions compared to their truncated counterparts.

5. Next, we will attempt to fine tune the basic models on the full datasets, and determine if there is any improvement in performance

Running GridSearchCV using StratifiedKFold cross validation in an attempt to improve model performance

# On Full CountVectorized Set

ML_Tuned_Models(X_train_cvfull, X_test_cvfull, y_train_cvfull, y_test_cvfull)

 model: RanForCls
 best parameters: 
{'ccp_alpha': 0.001, 'criterion': 'entropy', 'max_depth': 12, 'max_features': 'sqrt'}
 model: AdaBoosCls
 best parameters: 
{'learning_rate': 0.01, 'n_estimators': 10}
 model: LogReg
 best parameters: 
{'C': 1, 'penalty': 'l2'}
 model: KNN
 best parameters: 
{'metric': 'euclidean', 'n_neighbors': 3, 'weights': 'uniform'}
 model: Multinomial NB
 best parameters: 
{'alpha': 0.5, 'fit_prior': False}

	model	train best score	test best score
0	RanForCls	0.984756	0.409639
1	AdaBoosCls	0.390244	0.361446
2	LogReg	0.966463	0.421687
3	KNN	0.615854	0.337349
4	Multinomial NB	0.704268	0.397590

# On Full TFIDF Set

ML_Tuned_Models(X_train_tfidffull, X_test_tfidffull, y_train_tfidffull, y_test_tfidffull)

 model: RanForCls
 best parameters: 
{'ccp_alpha': 0.01, 'criterion': 'entropy', 'max_depth': 12, 'max_features': 'log2'}
 model: AdaBoosCls
 best parameters: 
{'learning_rate': 0.01, 'n_estimators': 10}
 model: LogReg
 best parameters: 
{'C': 10, 'penalty': 'l2'}
 model: KNN
 best parameters: 
{'metric': 'manhattan', 'n_neighbors': 9, 'weights': 'distance'}
 model: Multinomial NB
 best parameters: 
{'alpha': 1.1, 'fit_prior': True}

	model	train best score	test best score
0	RanForCls	0.954268	0.433735
1	AdaBoosCls	0.390244	0.361446
2	LogReg	0.969512	0.421687
3	KNN	1.000000	0.433735
4	Multinomial NB	0.557927	0.469880

# On Full Word2Vec Set

ML_Tuned_Models_without_MNB(X_train_wvfull, X_test_wvfull, y_train_wvfull, y_test_wvfull)

 model: RanForCls
 best parameters: 
{'ccp_alpha': 0.001, 'criterion': 'gini', 'max_depth': 11, 'max_features': 'sqrt'}
 model: AdaBoosCls
 best parameters: 
{'learning_rate': 0.01, 'n_estimators': 10}
 model: LogReg
 best parameters: 
{'C': 1, 'penalty': 'l2'}
 model: KNN
 best parameters: 
{'metric': 'euclidean', 'n_neighbors': 17, 'weights': 'uniform'}

	model	train best score	test best score
0	RanForCls	1.000000	0.361446
1	AdaBoosCls	0.390244	0.361446
2	LogReg	0.481707	0.421687
3	KNN	0.475610	0.445783

# Defining a function to train data on the tuned full CountVectorized dataset

def ML_Models_CVfull(X_train, X_test, y_train, y_test):

  models={
      "LogisticRegression":LogisticRegression(multi_class='multinomial', C = 1, penalty = 'l2', solver='saga', max_iter=10000,random_state = seed),
      "Multinomial NB": MultinomialNB(alpha = 0.5, fit_prior = False),
      "KNearestNeighbors": KNeighborsClassifier(metric = 'euclidean', n_neighbors = 3, weights = 'uniform'),
      "DecisionTreeClassifier":DecisionTreeClassifier(criterion='entropy',class_weight = 'balanced', max_depth = 6,random_state = seed, min_samples_leaf = 5),
      "RandomForestClassifier":RandomForestClassifier(ccp_alpha = 0.001, class_weight = 'balanced',criterion = 'entropy', n_estimators=100, max_features = 'sqrt', max_depth = 12,random_state = seed),
      "AdaBoostClassifier":AdaBoostClassifier(random_state = seed, learning_rate = 0.01, n_estimators = 10),
      "GradientBoostClassifier":GradientBoostingClassifier(random_state = seed),
      'XGBoostClassifier': xgb.XGBClassifier(colsample_bytree= 0.6, gamma= 0.5, max_depth= 5, min_child_weight= 1, subsample= 0.8,n_estimators=100)
  }



  names = []
  train_scores = []
  test_scores = []

  for name, model in models.items():
    model.fit(X_train, y_train)
    result_train = accuracy_score(y_train, model.predict(X_train))
    result_test = accuracy_score(y_test, model.predict(X_test))
    names.append(name)
    train_scores.append(result_train) # Appending the test scores to the list
    test_scores.append(result_test) # Appending the test scores to the list
    result_df = pd.DataFrame({'model': names, 'Train accuracy': train_scores, 'Test accuracy': test_scores})

    cm = confusion_matrix(y_test, model.predict(X_test))

    fig, ax = plt.subplots(nrows = 1, ncols = 1, figsize = (5,4))
    sns.heatmap(cm,
                annot=True,
                fmt='g',
                xticklabels=['I','II','III','IV','V'],
                yticklabels=['I','II','III','IV','V'], ax = ax)
    ax.set_ylabel('Prediction', fontsize = 11)
    ax.set_xlabel('Actual',fontsize = 11)
    ax.set_title(f'Confusion Matrix: {name} model',fontsize = 14)
    plt.show()

  return result_df

# Defining a function to train data on the tuned TFIDF dataset

def ML_Models_tfidffull(X_train, X_test, y_train, y_test):

  models={
      "LogisticRegression":LogisticRegression(multi_class='multinomial', C = 10, penalty = 'l2', solver='saga', max_iter=10000,random_state = seed),
      "Multinomial NB": MultinomialNB(alpha = 1.1, fit_prior = True),
      "KNearestNeighbors": KNeighborsClassifier(metric = 'manhattan', n_neighbors = 9, weights = 'distance'),
      "DecisionTreeClassifier":DecisionTreeClassifier(criterion='entropy',class_weight = 'balanced', max_depth = 6,random_state = seed, min_samples_leaf = 5),
      "RandomForestClassifier":RandomForestClassifier(ccp_alpha = 0.01, class_weight = 'balanced',criterion = 'entropy', n_estimators=100, max_features = 'log2', max_depth = 12,random_state = seed),
      "AdaBoostClassifier":AdaBoostClassifier(random_state = seed, learning_rate = 0.01, n_estimators = 10),
      "GradientBoostClassifier":GradientBoostingClassifier(random_state = seed),
      'XGBoostClassifier': xgb.XGBClassifier(colsample_bytree= 0.6, gamma= 0.5, max_depth= 5, min_child_weight= 1, subsample= 0.8,n_estimators=100)
  }


  names = []
  train_scores = []
  test_scores = []

  for name, model in models.items():
    model.fit(X_train, y_train)
    result_train = accuracy_score(y_train, model.predict(X_train))
    result_test = accuracy_score(y_test, model.predict(X_test))
    names.append(name)
    train_scores.append(result_train) # Appending the test scores to the list
    test_scores.append(result_test) # Appending the test scores to the list
    result_df = pd.DataFrame({'model': names, 'Train accuracy': train_scores, 'Test accuracy': test_scores})

    cm = confusion_matrix(y_test, model.predict(X_test))

    fig, ax = plt.subplots(nrows = 1, ncols = 1, figsize = (5,4))
    sns.heatmap(cm,
                annot=True,
                fmt='g',
                xticklabels=['I','II','III','IV','V'],
                yticklabels=['I','II','III','IV','V'], ax = ax)
    ax.set_ylabel('Prediction', fontsize = 11)
    ax.set_xlabel('Actual',fontsize = 11)
    ax.set_title(f'Confusion Matrix: {name} model',fontsize = 14)
    plt.show()

  return result_df

# Defining a function to train data on the tuned Word2Vec dataset

def ML_Models_wvfull(X_train, X_test, y_train, y_test):

  models={
      "LogisticRegression":LogisticRegression(multi_class='multinomial', C = 1, penalty = 'l2', solver='saga', max_iter=10000,random_state = seed),
    #   "Multinomial NB": MultinomialNB(alpha = 0.5, fit_prior = True),
      "KNearestNeighbors": KNeighborsClassifier(metric = 'euclidean', n_neighbors = 17, weights = 'uniform'),
      "DecisionTreeClassifier":DecisionTreeClassifier(criterion='entropy',class_weight = 'balanced', max_depth = 6,random_state = seed, min_samples_leaf = 5),
      "RandomForestClassifier":RandomForestClassifier(ccp_alpha = 0.001, class_weight = 'balanced',criterion = 'gini', n_estimators=100, max_features = 'sqrt', max_depth = 11,random_state = seed),
      "AdaBoostClassifier":AdaBoostClassifier(random_state = seed, learning_rate = 0.01, n_estimators = 10),
      "GradientBoostClassifier":GradientBoostingClassifier(random_state = seed),
      'XGBoostClassifier': xgb.XGBClassifier(colsample_bytree= 0.6, gamma= 0.5, max_depth= 5, min_child_weight= 1, subsample= 0.8,n_estimators=100)
  }


  names = []
  train_scores = []
  test_scores = []

  for name, model in models.items():
    model.fit(X_train, y_train)
    result_train = accuracy_score(y_train, model.predict(X_train))
    result_test = accuracy_score(y_test, model.predict(X_test))
    names.append(name)
    train_scores.append(result_train) # Appending the test scores to the list
    test_scores.append(result_test) # Appending the test scores to the list
    result_df = pd.DataFrame({'model': names, 'Train accuracy': train_scores, 'Test accuracy': test_scores})

    cm = confusion_matrix(y_test, model.predict(X_test))

    fig, ax = plt.subplots(nrows = 1, ncols = 1, figsize = (5,4))
    sns.heatmap(cm,
                annot=True,
                fmt='g',
                xticklabels=['I','II','III','IV','V'],
                yticklabels=['I','II','III','IV','V'], ax = ax)
    ax.set_ylabel('Prediction', fontsize = 11)
    ax.set_xlabel('Actual',fontsize = 11)
    ax.set_title(f'Confusion Matrix: {name} model',fontsize = 14)
    plt.show()

  return result_df

# Training and testing tuned full CountVectorized Datasets

ML_Models_CVfull(X_train_cvfull, X_test_cvfull, y_train_cvfull, y_test_cvfull)

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.966463	0.421687
1	Multinomial NB	0.704268	0.397590
2	KNearestNeighbors	0.615854	0.337349
3	DecisionTreeClassifier	0.506098	0.373494
4	RandomForestClassifier	0.987805	0.457831
5	AdaBoostClassifier	0.390244	0.361446
6	GradientBoostClassifier	0.981707	0.313253
7	XGBoostClassifier	0.966463	0.433735

# Training and testing tuned full TFIDF Datasets

ML_Models_tfidffull(X_train_tfidffull, X_test_tfidffull, y_train_tfidffull, y_test_tfidffull)

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.969512	0.421687
1	Multinomial NB	0.557927	0.469880
2	KNearestNeighbors	1.000000	0.433735
3	DecisionTreeClassifier	0.472561	0.397590
4	RandomForestClassifier	0.951220	0.445783
5	AdaBoostClassifier	0.390244	0.361446
6	GradientBoostClassifier	1.000000	0.385542
7	XGBoostClassifier	1.000000	0.385542

# Training and testing tuned full Word2Vec Datasets

ML_Models_wvfull(X_train_wvfull, X_test_wvfull, y_train_wvfull, y_test_wvfull)

	model	Train accuracy	Test accuracy
0	LogisticRegression	0.481707	0.421687
1	KNearestNeighbors	0.475610	0.445783
2	DecisionTreeClassifier	0.554878	0.216867
3	RandomForestClassifier	1.000000	0.385542
4	AdaBoostClassifier	0.390244	0.361446
5	GradientBoostClassifier	1.000000	0.361446
6	XGBoostClassifier	1.000000	0.373494

Neural Network Classifier-

There are various concepts of number of neurons in input and hidden layers of a neural network.

a. The number of neurons in the input layer is equal to the number of features in the data.

b. The number of hidden neurons should be 2/3 the size of the input layer, plus the size of the output layer.

In our case, considered tokenized data sample.

Number of neurons in input layer= 200 Number of neurons in hidden layer= 130

Applying ANN function on CountVectorizer dataset-

NN_Model(X_train_cv, X_test_cv, y_train_cv, y_test_cv)

Model: "sequential_10"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_30 (Dense)            (None, 150)               30150     
                                                                 
 dropout_10 (Dropout)        (None, 150)               0         
                                                                 
 dense_31 (Dense)            (None, 50)                7550      
                                                                 
 dense_32 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 78ms/step - loss: 1.6002 - accuracy: 0.2366 - val_loss: 1.5481 - val_accuracy: 0.3182
Epoch 2/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4489 - accuracy: 0.3969 - val_loss: 1.5154 - val_accuracy: 0.3333
Epoch 3/100
6/6 [==============================] - 0s 13ms/step - loss: 1.3575 - accuracy: 0.4466 - val_loss: 1.4999 - val_accuracy: 0.3030
Epoch 4/100
6/6 [==============================] - 0s 16ms/step - loss: 1.2804 - accuracy: 0.5191 - val_loss: 1.4949 - val_accuracy: 0.3636
Epoch 5/100
6/6 [==============================] - 0s 14ms/step - loss: 1.2206 - accuracy: 0.5229 - val_loss: 1.4840 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 14ms/step - loss: 1.1703 - accuracy: 0.5611 - val_loss: 1.4707 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 11ms/step - loss: 1.0924 - accuracy: 0.6069 - val_loss: 1.4603 - val_accuracy: 0.3485
Epoch 8/100
6/6 [==============================] - 0s 11ms/step - loss: 1.0137 - accuracy: 0.6374 - val_loss: 1.4511 - val_accuracy: 0.3939
Epoch 9/100
6/6 [==============================] - 0s 12ms/step - loss: 0.9316 - accuracy: 0.6794 - val_loss: 1.4455 - val_accuracy: 0.3939
Epoch 10/100
6/6 [==============================] - 0s 14ms/step - loss: 0.8671 - accuracy: 0.7405 - val_loss: 1.4487 - val_accuracy: 0.3939
Epoch 11/100
6/6 [==============================] - 0s 10ms/step - loss: 0.8246 - accuracy: 0.7481 - val_loss: 1.4543 - val_accuracy: 0.3939
Epoch 12/100
6/6 [==============================] - 0s 11ms/step - loss: 0.7531 - accuracy: 0.7824 - val_loss: 1.4480 - val_accuracy: 0.3788
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 5ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 5ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.746951	0.385542	0.743055	0.355307

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(10):
  result=NN_Model(X_train_cv, X_test_cv, y_train_cv, y_test_cv)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('F1 score')
plt.ylabel('F1 score')
plt.xlabel('epoch')
plt.show()

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense (Dense)               (None, 150)               30150     
                                                                 
 dropout (Dropout)           (None, 150)               0         
                                                                 
 dense_1 (Dense)             (None, 50)                7550      
                                                                 
 dense_2 (Dense)             (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 43ms/step - loss: 1.6172 - accuracy: 0.2099 - val_loss: 1.5400 - val_accuracy: 0.3182
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4621 - accuracy: 0.3893 - val_loss: 1.4949 - val_accuracy: 0.3182
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3857 - accuracy: 0.4160 - val_loss: 1.4707 - val_accuracy: 0.3182
Epoch 4/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2972 - accuracy: 0.4771 - val_loss: 1.4597 - val_accuracy: 0.3030
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2537 - accuracy: 0.5000 - val_loss: 1.4485 - val_accuracy: 0.3182
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1435 - accuracy: 0.6145 - val_loss: 1.4386 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0916 - accuracy: 0.6527 - val_loss: 1.4279 - val_accuracy: 0.3182
Epoch 8/100
6/6 [==============================] - 0s 6ms/step - loss: 1.0129 - accuracy: 0.6718 - val_loss: 1.4131 - val_accuracy: 0.3485
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9583 - accuracy: 0.7252 - val_loss: 1.4015 - val_accuracy: 0.3333
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8619 - accuracy: 0.7481 - val_loss: 1.4004 - val_accuracy: 0.3788
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8029 - accuracy: 0.7977 - val_loss: 1.4046 - val_accuracy: 0.3939
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7410 - accuracy: 0.8130 - val_loss: 1.3978 - val_accuracy: 0.3788
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 0.6751 - accuracy: 0.8168 - val_loss: 1.4025 - val_accuracy: 0.3788
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 0.5976 - accuracy: 0.8626 - val_loss: 1.4222 - val_accuracy: 0.3788
Epoch 15/100
6/6 [==============================] - 0s 7ms/step - loss: 0.5455 - accuracy: 0.8931 - val_loss: 1.4498 - val_accuracy: 0.3939
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step
Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_3 (Dense)             (None, 150)               30150     
                                                                 
 dropout_1 (Dropout)         (None, 150)               0         
                                                                 
 dense_4 (Dense)             (None, 50)                7550      
                                                                 
 dense_5 (Dense)             (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 57ms/step - loss: 1.6900 - accuracy: 0.1756 - val_loss: 1.6266 - val_accuracy: 0.2273
Epoch 2/100
6/6 [==============================] - 0s 11ms/step - loss: 1.5616 - accuracy: 0.2939 - val_loss: 1.5594 - val_accuracy: 0.3030
Epoch 3/100
6/6 [==============================] - 0s 10ms/step - loss: 1.4698 - accuracy: 0.4160 - val_loss: 1.5208 - val_accuracy: 0.3030
Epoch 4/100
6/6 [==============================] - 0s 10ms/step - loss: 1.4134 - accuracy: 0.4122 - val_loss: 1.4930 - val_accuracy: 0.3485
Epoch 5/100
6/6 [==============================] - 0s 10ms/step - loss: 1.3433 - accuracy: 0.5191 - val_loss: 1.4678 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 10ms/step - loss: 1.2882 - accuracy: 0.5115 - val_loss: 1.4492 - val_accuracy: 0.3485
Epoch 7/100
6/6 [==============================] - 0s 10ms/step - loss: 1.2291 - accuracy: 0.5534 - val_loss: 1.4357 - val_accuracy: 0.3636
Epoch 8/100
6/6 [==============================] - 0s 9ms/step - loss: 1.1452 - accuracy: 0.6641 - val_loss: 1.4238 - val_accuracy: 0.3636
Epoch 9/100
6/6 [==============================] - 0s 11ms/step - loss: 1.0815 - accuracy: 0.6641 - val_loss: 1.4138 - val_accuracy: 0.3485
Epoch 10/100
6/6 [==============================] - 0s 9ms/step - loss: 1.0011 - accuracy: 0.7214 - val_loss: 1.4055 - val_accuracy: 0.3182
Epoch 11/100
6/6 [==============================] - 0s 9ms/step - loss: 0.9472 - accuracy: 0.7366 - val_loss: 1.3981 - val_accuracy: 0.3636
Epoch 12/100
6/6 [==============================] - 0s 10ms/step - loss: 0.8672 - accuracy: 0.7672 - val_loss: 1.3859 - val_accuracy: 0.3788
Epoch 13/100
6/6 [==============================] - 0s 9ms/step - loss: 0.7861 - accuracy: 0.8053 - val_loss: 1.3809 - val_accuracy: 0.3788
Epoch 14/100
6/6 [==============================] - 0s 10ms/step - loss: 0.7329 - accuracy: 0.7977 - val_loss: 1.3877 - val_accuracy: 0.3788
Epoch 15/100
6/6 [==============================] - 0s 14ms/step - loss: 0.6703 - accuracy: 0.8702 - val_loss: 1.4124 - val_accuracy: 0.3636
Epoch 16/100
6/6 [==============================] - 0s 10ms/step - loss: 0.5954 - accuracy: 0.8817 - val_loss: 1.4431 - val_accuracy: 0.3788
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_2"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_6 (Dense)             (None, 150)               30150     
                                                                 
 dropout_2 (Dropout)         (None, 150)               0         
                                                                 
 dense_7 (Dense)             (None, 50)                7550      
                                                                 
 dense_8 (Dense)             (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 65ms/step - loss: 1.6376 - accuracy: 0.1870 - val_loss: 1.5778 - val_accuracy: 0.2727
Epoch 2/100
6/6 [==============================] - 0s 9ms/step - loss: 1.4639 - accuracy: 0.4542 - val_loss: 1.5350 - val_accuracy: 0.3030
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4056 - accuracy: 0.4580 - val_loss: 1.5078 - val_accuracy: 0.3485
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3069 - accuracy: 0.5305 - val_loss: 1.4966 - val_accuracy: 0.3182
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2351 - accuracy: 0.5573 - val_loss: 1.4869 - val_accuracy: 0.3030
Epoch 6/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1685 - accuracy: 0.5802 - val_loss: 1.4786 - val_accuracy: 0.3030
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1038 - accuracy: 0.6221 - val_loss: 1.4727 - val_accuracy: 0.3182
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0287 - accuracy: 0.6718 - val_loss: 1.4643 - val_accuracy: 0.3636
Epoch 9/100
6/6 [==============================] - 0s 10ms/step - loss: 0.9507 - accuracy: 0.7290 - val_loss: 1.4532 - val_accuracy: 0.3788
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8890 - accuracy: 0.7481 - val_loss: 1.4562 - val_accuracy: 0.3788
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8017 - accuracy: 0.7824 - val_loss: 1.4630 - val_accuracy: 0.3485
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7298 - accuracy: 0.8244 - val_loss: 1.4547 - val_accuracy: 0.3636
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_3"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_9 (Dense)             (None, 150)               30150     
                                                                 
 dropout_3 (Dropout)         (None, 150)               0         
                                                                 
 dense_10 (Dense)            (None, 50)                7550      
                                                                 
 dense_11 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 39ms/step - loss: 1.6286 - accuracy: 0.2748 - val_loss: 1.5387 - val_accuracy: 0.2424
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4526 - accuracy: 0.3931 - val_loss: 1.4868 - val_accuracy: 0.3788
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3755 - accuracy: 0.4046 - val_loss: 1.4729 - val_accuracy: 0.3636
Epoch 4/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2978 - accuracy: 0.4542 - val_loss: 1.4689 - val_accuracy: 0.4091
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2338 - accuracy: 0.5496 - val_loss: 1.4606 - val_accuracy: 0.4091
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1824 - accuracy: 0.5153 - val_loss: 1.4486 - val_accuracy: 0.4242
Epoch 7/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1202 - accuracy: 0.6412 - val_loss: 1.4372 - val_accuracy: 0.4394
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0542 - accuracy: 0.6832 - val_loss: 1.4234 - val_accuracy: 0.3939
Epoch 9/100
6/6 [==============================] - 0s 8ms/step - loss: 0.9926 - accuracy: 0.6985 - val_loss: 1.4142 - val_accuracy: 0.3939
Epoch 10/100
6/6 [==============================] - 0s 8ms/step - loss: 0.9133 - accuracy: 0.7290 - val_loss: 1.4213 - val_accuracy: 0.3788
Epoch 11/100
6/6 [==============================] - 0s 8ms/step - loss: 0.8782 - accuracy: 0.7328 - val_loss: 1.4350 - val_accuracy: 0.3636
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8089 - accuracy: 0.8015 - val_loss: 1.4275 - val_accuracy: 0.3788
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_4"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_12 (Dense)            (None, 150)               30150     
                                                                 
 dropout_4 (Dropout)         (None, 150)               0         
                                                                 
 dense_13 (Dense)            (None, 50)                7550      
                                                                 
 dense_14 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 39ms/step - loss: 1.5863 - accuracy: 0.2901 - val_loss: 1.5684 - val_accuracy: 0.2424
Epoch 2/100
6/6 [==============================] - 0s 8ms/step - loss: 1.4623 - accuracy: 0.3779 - val_loss: 1.5334 - val_accuracy: 0.2576
Epoch 3/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3821 - accuracy: 0.4084 - val_loss: 1.5194 - val_accuracy: 0.2576
Epoch 4/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3018 - accuracy: 0.4389 - val_loss: 1.5226 - val_accuracy: 0.2727
Epoch 5/100
6/6 [==============================] - 0s 11ms/step - loss: 1.2275 - accuracy: 0.5115 - val_loss: 1.5218 - val_accuracy: 0.2879
Epoch 6/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1866 - accuracy: 0.5496 - val_loss: 1.5153 - val_accuracy: 0.2727
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1063 - accuracy: 0.5840 - val_loss: 1.5070 - val_accuracy: 0.2727
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0422 - accuracy: 0.6756 - val_loss: 1.4980 - val_accuracy: 0.2879
Epoch 9/100
6/6 [==============================] - 0s 6ms/step - loss: 0.9771 - accuracy: 0.7023 - val_loss: 1.4894 - val_accuracy: 0.3182
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8923 - accuracy: 0.7519 - val_loss: 1.4959 - val_accuracy: 0.3485
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8399 - accuracy: 0.7824 - val_loss: 1.5141 - val_accuracy: 0.3333
Epoch 12/100
6/6 [==============================] - 0s 6ms/step - loss: 0.7630 - accuracy: 0.8015 - val_loss: 1.5186 - val_accuracy: 0.3636
11/11 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_5"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_15 (Dense)            (None, 150)               30150     
                                                                 
 dropout_5 (Dropout)         (None, 150)               0         
                                                                 
 dense_16 (Dense)            (None, 50)                7550      
                                                                 
 dense_17 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 39ms/step - loss: 1.6478 - accuracy: 0.1985 - val_loss: 1.5816 - val_accuracy: 0.3182
Epoch 2/100
6/6 [==============================] - 0s 10ms/step - loss: 1.5087 - accuracy: 0.3473 - val_loss: 1.5408 - val_accuracy: 0.3182
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4114 - accuracy: 0.3855 - val_loss: 1.5221 - val_accuracy: 0.3030
Epoch 4/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3553 - accuracy: 0.3893 - val_loss: 1.5145 - val_accuracy: 0.2727
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2845 - accuracy: 0.4237 - val_loss: 1.5084 - val_accuracy: 0.2727
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2230 - accuracy: 0.5038 - val_loss: 1.5051 - val_accuracy: 0.2879
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1604 - accuracy: 0.5649 - val_loss: 1.5031 - val_accuracy: 0.3030
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1082 - accuracy: 0.6450 - val_loss: 1.4981 - val_accuracy: 0.2879
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0318 - accuracy: 0.6870 - val_loss: 1.4888 - val_accuracy: 0.3030
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9499 - accuracy: 0.7519 - val_loss: 1.4900 - val_accuracy: 0.3333
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8772 - accuracy: 0.7824 - val_loss: 1.4930 - val_accuracy: 0.3333
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8159 - accuracy: 0.7748 - val_loss: 1.4840 - val_accuracy: 0.3636
Epoch 13/100
6/6 [==============================] - 0s 8ms/step - loss: 0.7430 - accuracy: 0.8282 - val_loss: 1.4889 - val_accuracy: 0.3939
Epoch 14/100
6/6 [==============================] - 0s 10ms/step - loss: 0.6792 - accuracy: 0.8511 - val_loss: 1.5110 - val_accuracy: 0.3788
Epoch 15/100
6/6 [==============================] - 0s 7ms/step - loss: 0.5954 - accuracy: 0.8740 - val_loss: 1.5448 - val_accuracy: 0.3788
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_6"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_18 (Dense)            (None, 150)               30150     
                                                                 
 dropout_6 (Dropout)         (None, 150)               0         
                                                                 
 dense_19 (Dense)            (None, 50)                7550      
                                                                 
 dense_20 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 38ms/step - loss: 1.5631 - accuracy: 0.2824 - val_loss: 1.5403 - val_accuracy: 0.3030
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4292 - accuracy: 0.3893 - val_loss: 1.5164 - val_accuracy: 0.3030
Epoch 3/100
6/6 [==============================] - 0s 6ms/step - loss: 1.3597 - accuracy: 0.4198 - val_loss: 1.5039 - val_accuracy: 0.3030
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3051 - accuracy: 0.4542 - val_loss: 1.4987 - val_accuracy: 0.2879
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2222 - accuracy: 0.5420 - val_loss: 1.4869 - val_accuracy: 0.2727
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1732 - accuracy: 0.5344 - val_loss: 1.4733 - val_accuracy: 0.3030
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0899 - accuracy: 0.6069 - val_loss: 1.4599 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0237 - accuracy: 0.6908 - val_loss: 1.4461 - val_accuracy: 0.3485
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9699 - accuracy: 0.7214 - val_loss: 1.4284 - val_accuracy: 0.3333
Epoch 10/100
6/6 [==============================] - 0s 8ms/step - loss: 0.9002 - accuracy: 0.7481 - val_loss: 1.4256 - val_accuracy: 0.2727
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8213 - accuracy: 0.7901 - val_loss: 1.4413 - val_accuracy: 0.3182
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7587 - accuracy: 0.8092 - val_loss: 1.4441 - val_accuracy: 0.3333
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 0.6967 - accuracy: 0.8321 - val_loss: 1.4473 - val_accuracy: 0.3030
11/11 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 2ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_7"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_21 (Dense)            (None, 150)               30150     
                                                                 
 dropout_7 (Dropout)         (None, 150)               0         
                                                                 
 dense_22 (Dense)            (None, 50)                7550      
                                                                 
 dense_23 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 37ms/step - loss: 1.6248 - accuracy: 0.2405 - val_loss: 1.5193 - val_accuracy: 0.3333
Epoch 2/100
6/6 [==============================] - 0s 11ms/step - loss: 1.4727 - accuracy: 0.3626 - val_loss: 1.4925 - val_accuracy: 0.3182
Epoch 3/100
6/6 [==============================] - 0s 12ms/step - loss: 1.3910 - accuracy: 0.3588 - val_loss: 1.4834 - val_accuracy: 0.3333
Epoch 4/100
6/6 [==============================] - 0s 14ms/step - loss: 1.3062 - accuracy: 0.4008 - val_loss: 1.4846 - val_accuracy: 0.3333
Epoch 5/100
6/6 [==============================] - 0s 9ms/step - loss: 1.2542 - accuracy: 0.4313 - val_loss: 1.4792 - val_accuracy: 0.3182
Epoch 6/100
6/6 [==============================] - 0s 14ms/step - loss: 1.2010 - accuracy: 0.4962 - val_loss: 1.4731 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 10ms/step - loss: 1.1196 - accuracy: 0.5687 - val_loss: 1.4666 - val_accuracy: 0.3030
Epoch 8/100
6/6 [==============================] - 0s 9ms/step - loss: 1.0540 - accuracy: 0.6183 - val_loss: 1.4579 - val_accuracy: 0.2576
Epoch 9/100
6/6 [==============================] - 0s 15ms/step - loss: 1.0103 - accuracy: 0.6756 - val_loss: 1.4480 - val_accuracy: 0.2879
Epoch 10/100
6/6 [==============================] - 0s 14ms/step - loss: 0.9162 - accuracy: 0.7672 - val_loss: 1.4519 - val_accuracy: 0.2879
Epoch 11/100
6/6 [==============================] - 0s 10ms/step - loss: 0.8619 - accuracy: 0.7595 - val_loss: 1.4665 - val_accuracy: 0.3333
Epoch 12/100
6/6 [==============================] - 0s 9ms/step - loss: 0.7879 - accuracy: 0.8015 - val_loss: 1.4630 - val_accuracy: 0.3485
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_8"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_24 (Dense)            (None, 150)               30150     
                                                                 
 dropout_8 (Dropout)         (None, 150)               0         
                                                                 
 dense_25 (Dense)            (None, 50)                7550      
                                                                 
 dense_26 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 58ms/step - loss: 1.5196 - accuracy: 0.3015 - val_loss: 1.5146 - val_accuracy: 0.2576
Epoch 2/100
6/6 [==============================] - 0s 14ms/step - loss: 1.4065 - accuracy: 0.3740 - val_loss: 1.5016 - val_accuracy: 0.2424
Epoch 3/100
6/6 [==============================] - 0s 10ms/step - loss: 1.3386 - accuracy: 0.3893 - val_loss: 1.4905 - val_accuracy: 0.2424
Epoch 4/100
6/6 [==============================] - 0s 11ms/step - loss: 1.2731 - accuracy: 0.4733 - val_loss: 1.4880 - val_accuracy: 0.2273
Epoch 5/100
6/6 [==============================] - 0s 10ms/step - loss: 1.2148 - accuracy: 0.4924 - val_loss: 1.4806 - val_accuracy: 0.2424
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1511 - accuracy: 0.5534 - val_loss: 1.4750 - val_accuracy: 0.2424
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0631 - accuracy: 0.6412 - val_loss: 1.4716 - val_accuracy: 0.2727
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0046 - accuracy: 0.6947 - val_loss: 1.4687 - val_accuracy: 0.2424
Epoch 9/100
6/6 [==============================] - 0s 8ms/step - loss: 0.9380 - accuracy: 0.6947 - val_loss: 1.4703 - val_accuracy: 0.2576
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8802 - accuracy: 0.7557 - val_loss: 1.4771 - val_accuracy: 0.2727
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8071 - accuracy: 0.7939 - val_loss: 1.4864 - val_accuracy: 0.2727
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step
Model: "sequential_9"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_27 (Dense)            (None, 150)               30150     
                                                                 
 dropout_9 (Dropout)         (None, 150)               0         
                                                                 
 dense_28 (Dense)            (None, 50)                7550      
                                                                 
 dense_29 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 35ms/step - loss: 1.5642 - accuracy: 0.2557 - val_loss: 1.5391 - val_accuracy: 0.2727
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4388 - accuracy: 0.3893 - val_loss: 1.4979 - val_accuracy: 0.3030
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3583 - accuracy: 0.4618 - val_loss: 1.4790 - val_accuracy: 0.3030
Epoch 4/100
6/6 [==============================] - 0s 6ms/step - loss: 1.2743 - accuracy: 0.5076 - val_loss: 1.4695 - val_accuracy: 0.3182
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1991 - accuracy: 0.5954 - val_loss: 1.4608 - val_accuracy: 0.2879
Epoch 6/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1497 - accuracy: 0.6107 - val_loss: 1.4517 - val_accuracy: 0.3182
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0744 - accuracy: 0.6718 - val_loss: 1.4425 - val_accuracy: 0.3030
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0240 - accuracy: 0.6985 - val_loss: 1.4298 - val_accuracy: 0.3182
Epoch 9/100
6/6 [==============================] - 0s 6ms/step - loss: 0.9568 - accuracy: 0.6985 - val_loss: 1.4194 - val_accuracy: 0.3485
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8952 - accuracy: 0.7634 - val_loss: 1.4211 - val_accuracy: 0.3333
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8122 - accuracy: 0.7977 - val_loss: 1.4286 - val_accuracy: 0.3485
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7681 - accuracy: 0.8053 - val_loss: 1.4121 - val_accuracy: 0.3485
Epoch 13/100
6/6 [==============================] - 0s 8ms/step - loss: 0.6758 - accuracy: 0.8511 - val_loss: 1.4163 - val_accuracy: 0.3333
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 0.6427 - accuracy: 0.8664 - val_loss: 1.4366 - val_accuracy: 0.3485
Epoch 15/100
6/6 [==============================] - 0s 12ms/step - loss: 0.5724 - accuracy: 0.8740 - val_loss: 1.4702 - val_accuracy: 0.3636
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step

NN_Model(X_train_cv_smote, X_test_cv, y_train_cv_smote, y_test_cv)

Model: "sequential_66"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_198 (Dense)           (None, 150)               30150     
                                                                 
 dropout_66 (Dropout)        (None, 150)               0         
                                                                 
 dense_199 (Dense)           (None, 50)                7550      
                                                                 
 dense_200 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 44ms/step - loss: 1.5970 - accuracy: 0.2636 - val_loss: 1.6877 - val_accuracy: 0.0727
Epoch 2/100
9/9 [==============================] - 0s 9ms/step - loss: 1.4549 - accuracy: 0.4273 - val_loss: 1.7643 - val_accuracy: 0.0182
Epoch 3/100
9/9 [==============================] - 0s 9ms/step - loss: 1.3378 - accuracy: 0.5091 - val_loss: 1.8058 - val_accuracy: 0.0182
Epoch 4/100
9/9 [==============================] - 0s 12ms/step - loss: 1.2342 - accuracy: 0.5636 - val_loss: 1.8210 - val_accuracy: 0.0182
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 5ms/step
18/18 [==============================] - 0s 6ms/step
3/3 [==============================] - 0s 5ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.541818	0.349398	0.471925	0.253779

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(10):
  result=NN_Model(X_train_cv_smote, X_test_cv, y_train_cv_smote, y_test_cv)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('F1 score')
plt.ylabel('F1 score')
plt.xlabel('epoch')
plt.show()

Model: "sequential_67"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_201 (Dense)           (None, 150)               30150     
                                                                 
 dropout_67 (Dropout)        (None, 150)               0         
                                                                 
 dense_202 (Dense)           (None, 50)                7550      
                                                                 
 dense_203 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 45ms/step - loss: 1.5794 - accuracy: 0.2432 - val_loss: 1.6871 - val_accuracy: 0.1000
Epoch 2/100
9/9 [==============================] - 0s 9ms/step - loss: 1.4602 - accuracy: 0.4136 - val_loss: 1.7886 - val_accuracy: 0.0545
Epoch 3/100
9/9 [==============================] - 0s 12ms/step - loss: 1.3404 - accuracy: 0.5091 - val_loss: 1.8387 - val_accuracy: 0.0273
Epoch 4/100
9/9 [==============================] - 0s 11ms/step - loss: 1.2434 - accuracy: 0.6068 - val_loss: 1.8369 - val_accuracy: 0.0364
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 5ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_68"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_204 (Dense)           (None, 150)               30150     
                                                                 
 dropout_68 (Dropout)        (None, 150)               0         
                                                                 
 dense_205 (Dense)           (None, 50)                7550      
                                                                 
 dense_206 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 45ms/step - loss: 1.6119 - accuracy: 0.1955 - val_loss: 1.6085 - val_accuracy: 0.0727
Epoch 2/100
9/9 [==============================] - 0s 9ms/step - loss: 1.4791 - accuracy: 0.3682 - val_loss: 1.7124 - val_accuracy: 0.0455
Epoch 3/100
9/9 [==============================] - 0s 11ms/step - loss: 1.3820 - accuracy: 0.4682 - val_loss: 1.7646 - val_accuracy: 0.0545
Epoch 4/100
9/9 [==============================] - 0s 9ms/step - loss: 1.2967 - accuracy: 0.5295 - val_loss: 1.7874 - val_accuracy: 0.0727
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 6ms/step
18/18 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 6ms/step
Model: "sequential_69"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_207 (Dense)           (None, 150)               30150     
                                                                 
 dropout_69 (Dropout)        (None, 150)               0         
                                                                 
 dense_208 (Dense)           (None, 50)                7550      
                                                                 
 dense_209 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 33ms/step - loss: 1.6156 - accuracy: 0.1909 - val_loss: 1.6581 - val_accuracy: 0.0364
Epoch 2/100
9/9 [==============================] - 0s 11ms/step - loss: 1.4731 - accuracy: 0.3523 - val_loss: 1.7717 - val_accuracy: 0.0091
Epoch 3/100
9/9 [==============================] - 0s 8ms/step - loss: 1.3726 - accuracy: 0.4136 - val_loss: 1.8271 - val_accuracy: 0.0091
Epoch 4/100
9/9 [==============================] - 0s 11ms/step - loss: 1.2722 - accuracy: 0.5295 - val_loss: 1.8197 - val_accuracy: 0.0273
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_70"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_210 (Dense)           (None, 150)               30150     
                                                                 
 dropout_70 (Dropout)        (None, 150)               0         
                                                                 
 dense_211 (Dense)           (None, 50)                7550      
                                                                 
 dense_212 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 28ms/step - loss: 1.5710 - accuracy: 0.2682 - val_loss: 1.7394 - val_accuracy: 0.0545
Epoch 2/100
9/9 [==============================] - 0s 6ms/step - loss: 1.4335 - accuracy: 0.4136 - val_loss: 1.8673 - val_accuracy: 0.0455
Epoch 3/100
9/9 [==============================] - 0s 7ms/step - loss: 1.3401 - accuracy: 0.4977 - val_loss: 1.8698 - val_accuracy: 0.0636
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.2368 - accuracy: 0.6136 - val_loss: 1.8223 - val_accuracy: 0.0636
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_71"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_213 (Dense)           (None, 150)               30150     
                                                                 
 dropout_71 (Dropout)        (None, 150)               0         
                                                                 
 dense_214 (Dense)           (None, 50)                7550      
                                                                 
 dense_215 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 26ms/step - loss: 1.5523 - accuracy: 0.2705 - val_loss: 1.7848 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 6ms/step - loss: 1.4245 - accuracy: 0.3955 - val_loss: 1.8918 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3331 - accuracy: 0.4795 - val_loss: 1.8866 - val_accuracy: 0.0091
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.2309 - accuracy: 0.5977 - val_loss: 1.8601 - val_accuracy: 0.0364
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_72"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_216 (Dense)           (None, 150)               30150     
                                                                 
 dropout_72 (Dropout)        (None, 150)               0         
                                                                 
 dense_217 (Dense)           (None, 50)                7550      
                                                                 
 dense_218 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 23ms/step - loss: 1.5873 - accuracy: 0.2227 - val_loss: 1.7276 - val_accuracy: 0.0091
Epoch 2/100
9/9 [==============================] - 0s 6ms/step - loss: 1.4624 - accuracy: 0.3500 - val_loss: 1.8249 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 8ms/step - loss: 1.3734 - accuracy: 0.4841 - val_loss: 1.8441 - val_accuracy: 0.0091
Epoch 4/100
9/9 [==============================] - 0s 7ms/step - loss: 1.2769 - accuracy: 0.5705 - val_loss: 1.8372 - val_accuracy: 0.0364
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_73"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_219 (Dense)           (None, 150)               30150     
                                                                 
 dropout_73 (Dropout)        (None, 150)               0         
                                                                 
 dense_220 (Dense)           (None, 50)                7550      
                                                                 
 dense_221 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 23ms/step - loss: 1.6412 - accuracy: 0.1864 - val_loss: 1.6472 - val_accuracy: 0.0364
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.5026 - accuracy: 0.3682 - val_loss: 1.7448 - val_accuracy: 0.0273
Epoch 3/100
9/9 [==============================] - 0s 8ms/step - loss: 1.4011 - accuracy: 0.4727 - val_loss: 1.7838 - val_accuracy: 0.0364
Epoch 4/100
9/9 [==============================] - 0s 7ms/step - loss: 1.3013 - accuracy: 0.6136 - val_loss: 1.7936 - val_accuracy: 0.0636
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_74"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_222 (Dense)           (None, 150)               30150     
                                                                 
 dropout_74 (Dropout)        (None, 150)               0         
                                                                 
 dense_223 (Dense)           (None, 50)                7550      
                                                                 
 dense_224 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 25ms/step - loss: 1.6080 - accuracy: 0.2500 - val_loss: 1.7626 - val_accuracy: 0.0455
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.4655 - accuracy: 0.3818 - val_loss: 1.8825 - val_accuracy: 0.0182
Epoch 3/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3739 - accuracy: 0.4727 - val_loss: 1.8952 - val_accuracy: 0.0364
Epoch 4/100
9/9 [==============================] - 0s 7ms/step - loss: 1.2758 - accuracy: 0.5932 - val_loss: 1.8485 - val_accuracy: 0.0273
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_75"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_225 (Dense)           (None, 150)               30150     
                                                                 
 dropout_75 (Dropout)        (None, 150)               0         
                                                                 
 dense_226 (Dense)           (None, 50)                7550      
                                                                 
 dense_227 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 26ms/step - loss: 1.5898 - accuracy: 0.2205 - val_loss: 1.7558 - val_accuracy: 0.0818
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.4575 - accuracy: 0.4091 - val_loss: 1.8179 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3691 - accuracy: 0.4841 - val_loss: 1.8489 - val_accuracy: 0.0182
Epoch 4/100
9/9 [==============================] - 0s 7ms/step - loss: 1.2645 - accuracy: 0.5955 - val_loss: 1.8574 - val_accuracy: 0.0182
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_76"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_228 (Dense)           (None, 150)               30150     
                                                                 
 dropout_76 (Dropout)        (None, 150)               0         
                                                                 
 dense_229 (Dense)           (None, 50)                7550      
                                                                 
 dense_230 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 59ms/step - loss: 1.5477 - accuracy: 0.2750 - val_loss: 1.8138 - val_accuracy: 0.0545
Epoch 2/100
9/9 [==============================] - 0s 15ms/step - loss: 1.4300 - accuracy: 0.4182 - val_loss: 1.9179 - val_accuracy: 0.0091
Epoch 3/100
9/9 [==============================] - 0s 13ms/step - loss: 1.3414 - accuracy: 0.5273 - val_loss: 1.9530 - val_accuracy: 0.0182
Epoch 4/100
9/9 [==============================] - 0s 15ms/step - loss: 1.2358 - accuracy: 0.5955 - val_loss: 1.9488 - val_accuracy: 0.0182
18/18 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 4ms/step
18/18 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 4ms/step

Applying ANN function on CountVectorizer full dataset-

NN_Model(X_train_cvfull, X_test_cvfull, y_train_cvfull, y_test_cvfull)

Model: "sequential_11"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_33 (Dense)            (None, 150)               33000     
                                                                 
 dropout_11 (Dropout)        (None, 150)               0         
                                                                 
 dense_34 (Dense)            (None, 50)                7550      
                                                                 
 dense_35 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 82ms/step - loss: 1.5672 - accuracy: 0.2863 - val_loss: 1.5310 - val_accuracy: 0.2879
Epoch 2/100
6/6 [==============================] - 0s 11ms/step - loss: 1.4348 - accuracy: 0.4160 - val_loss: 1.4898 - val_accuracy: 0.3333
Epoch 3/100
6/6 [==============================] - 0s 12ms/step - loss: 1.3207 - accuracy: 0.4809 - val_loss: 1.4560 - val_accuracy: 0.3636
Epoch 4/100
6/6 [==============================] - 0s 23ms/step - loss: 1.2489 - accuracy: 0.5115 - val_loss: 1.4331 - val_accuracy: 0.3485
Epoch 5/100
6/6 [==============================] - 0s 19ms/step - loss: 1.1525 - accuracy: 0.5992 - val_loss: 1.4080 - val_accuracy: 0.3182
Epoch 6/100
6/6 [==============================] - 0s 19ms/step - loss: 1.0705 - accuracy: 0.6412 - val_loss: 1.3826 - val_accuracy: 0.3485
Epoch 7/100
6/6 [==============================] - 0s 13ms/step - loss: 1.0036 - accuracy: 0.6794 - val_loss: 1.3587 - val_accuracy: 0.3788
Epoch 8/100
6/6 [==============================] - 0s 12ms/step - loss: 0.9243 - accuracy: 0.7214 - val_loss: 1.3374 - val_accuracy: 0.3636
Epoch 9/100
6/6 [==============================] - 0s 16ms/step - loss: 0.8576 - accuracy: 0.7214 - val_loss: 1.3233 - val_accuracy: 0.3636
Epoch 10/100
6/6 [==============================] - 0s 24ms/step - loss: 0.7774 - accuracy: 0.7672 - val_loss: 1.3189 - val_accuracy: 0.3939
Epoch 11/100
6/6 [==============================] - 0s 20ms/step - loss: 0.7276 - accuracy: 0.8092 - val_loss: 1.3100 - val_accuracy: 0.4091
Epoch 12/100
6/6 [==============================] - 0s 11ms/step - loss: 0.6520 - accuracy: 0.8244 - val_loss: 1.2991 - val_accuracy: 0.4697
Epoch 13/100
6/6 [==============================] - 0s 20ms/step - loss: 0.5755 - accuracy: 0.8588 - val_loss: 1.3155 - val_accuracy: 0.4545
Epoch 14/100
6/6 [==============================] - 0s 19ms/step - loss: 0.5060 - accuracy: 0.9046 - val_loss: 1.3454 - val_accuracy: 0.4394
Epoch 15/100
6/6 [==============================] - 0s 20ms/step - loss: 0.4579 - accuracy: 0.8855 - val_loss: 1.3840 - val_accuracy: 0.4394
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
11/11 [==============================] - 0s 5ms/step
3/3 [==============================] - 0s 5ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.847561	0.445783	0.846316	0.4275

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(10):
  result=NN_Model(X_train_cvfull, X_test_cvfull, y_train_cvfull, y_test_cvfull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('F1 score')
plt.ylabel('F1 score')
plt.xlabel('epoch')
plt.show()

Model: "sequential_12"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_36 (Dense)            (None, 150)               33000     
                                                                 
 dropout_12 (Dropout)        (None, 150)               0         
                                                                 
 dense_37 (Dense)            (None, 50)                7550      
                                                                 
 dense_38 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 70ms/step - loss: 1.6163 - accuracy: 0.1718 - val_loss: 1.5314 - val_accuracy: 0.3485
Epoch 2/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4517 - accuracy: 0.3588 - val_loss: 1.4925 - val_accuracy: 0.3333
Epoch 3/100
6/6 [==============================] - 0s 22ms/step - loss: 1.3824 - accuracy: 0.4160 - val_loss: 1.4740 - val_accuracy: 0.3333
Epoch 4/100
6/6 [==============================] - 0s 14ms/step - loss: 1.3105 - accuracy: 0.4466 - val_loss: 1.4588 - val_accuracy: 0.3636
Epoch 5/100
6/6 [==============================] - 0s 10ms/step - loss: 1.2350 - accuracy: 0.5534 - val_loss: 1.4400 - val_accuracy: 0.3788
Epoch 6/100
6/6 [==============================] - 0s 11ms/step - loss: 1.1728 - accuracy: 0.6145 - val_loss: 1.4212 - val_accuracy: 0.3939
Epoch 7/100
6/6 [==============================] - 0s 11ms/step - loss: 1.0973 - accuracy: 0.6374 - val_loss: 1.4017 - val_accuracy: 0.3939
Epoch 8/100
6/6 [==============================] - 0s 11ms/step - loss: 1.0195 - accuracy: 0.7328 - val_loss: 1.3855 - val_accuracy: 0.3939
Epoch 9/100
6/6 [==============================] - 0s 11ms/step - loss: 0.9555 - accuracy: 0.7176 - val_loss: 1.3720 - val_accuracy: 0.3939
Epoch 10/100
6/6 [==============================] - 0s 14ms/step - loss: 0.8854 - accuracy: 0.7824 - val_loss: 1.3662 - val_accuracy: 0.4394
Epoch 11/100
6/6 [==============================] - 0s 12ms/step - loss: 0.8340 - accuracy: 0.7748 - val_loss: 1.3652 - val_accuracy: 0.4091
Epoch 12/100
6/6 [==============================] - 0s 11ms/step - loss: 0.7573 - accuracy: 0.8053 - val_loss: 1.3540 - val_accuracy: 0.4697
Epoch 13/100
6/6 [==============================] - 0s 11ms/step - loss: 0.6806 - accuracy: 0.8282 - val_loss: 1.3574 - val_accuracy: 0.4545
Epoch 14/100
6/6 [==============================] - 0s 11ms/step - loss: 0.6266 - accuracy: 0.8588 - val_loss: 1.3715 - val_accuracy: 0.4697
Epoch 15/100
6/6 [==============================] - 0s 11ms/step - loss: 0.5425 - accuracy: 0.8779 - val_loss: 1.3889 - val_accuracy: 0.4545
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 5ms/step
Model: "sequential_13"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_39 (Dense)            (None, 150)               33000     
                                                                 
 dropout_13 (Dropout)        (None, 150)               0         
                                                                 
 dense_40 (Dense)            (None, 50)                7550      
                                                                 
 dense_41 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 60ms/step - loss: 1.5682 - accuracy: 0.2939 - val_loss: 1.5245 - val_accuracy: 0.3788
Epoch 2/100
6/6 [==============================] - 0s 10ms/step - loss: 1.4201 - accuracy: 0.4122 - val_loss: 1.4867 - val_accuracy: 0.3939
Epoch 3/100
6/6 [==============================] - 0s 10ms/step - loss: 1.3321 - accuracy: 0.4809 - val_loss: 1.4655 - val_accuracy: 0.3182
Epoch 4/100
6/6 [==============================] - 0s 10ms/step - loss: 1.2636 - accuracy: 0.5496 - val_loss: 1.4508 - val_accuracy: 0.3182
Epoch 5/100
6/6 [==============================] - 0s 10ms/step - loss: 1.1796 - accuracy: 0.5802 - val_loss: 1.4253 - val_accuracy: 0.3636
Epoch 6/100
6/6 [==============================] - 0s 9ms/step - loss: 1.1097 - accuracy: 0.6260 - val_loss: 1.3962 - val_accuracy: 0.3030
Epoch 7/100
6/6 [==============================] - 0s 9ms/step - loss: 1.0287 - accuracy: 0.6947 - val_loss: 1.3700 - val_accuracy: 0.2879
Epoch 8/100
6/6 [==============================] - 0s 10ms/step - loss: 0.9553 - accuracy: 0.7405 - val_loss: 1.3492 - val_accuracy: 0.3182
Epoch 9/100
6/6 [==============================] - 0s 10ms/step - loss: 0.8798 - accuracy: 0.7824 - val_loss: 1.3338 - val_accuracy: 0.3182
Epoch 10/100
6/6 [==============================] - 0s 10ms/step - loss: 0.7927 - accuracy: 0.8168 - val_loss: 1.3291 - val_accuracy: 0.3182
Epoch 11/100
6/6 [==============================] - 0s 10ms/step - loss: 0.7455 - accuracy: 0.8092 - val_loss: 1.3282 - val_accuracy: 0.3485
Epoch 12/100
6/6 [==============================] - 0s 10ms/step - loss: 0.6779 - accuracy: 0.8359 - val_loss: 1.3247 - val_accuracy: 0.3485
Epoch 13/100
6/6 [==============================] - 0s 10ms/step - loss: 0.6179 - accuracy: 0.8664 - val_loss: 1.3384 - val_accuracy: 0.3485
Epoch 14/100
6/6 [==============================] - 0s 10ms/step - loss: 0.5748 - accuracy: 0.8511 - val_loss: 1.3522 - val_accuracy: 0.3788
Epoch 15/100
6/6 [==============================] - 0s 10ms/step - loss: 0.4885 - accuracy: 0.8893 - val_loss: 1.3594 - val_accuracy: 0.3636
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_14"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_42 (Dense)            (None, 150)               33000     
                                                                 
 dropout_14 (Dropout)        (None, 150)               0         
                                                                 
 dense_43 (Dense)            (None, 50)                7550      
                                                                 
 dense_44 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 36ms/step - loss: 1.6026 - accuracy: 0.2061 - val_loss: 1.5131 - val_accuracy: 0.2879
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4184 - accuracy: 0.3626 - val_loss: 1.4692 - val_accuracy: 0.2879
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3086 - accuracy: 0.4122 - val_loss: 1.4470 - val_accuracy: 0.3030
Epoch 4/100
6/6 [==============================] - 0s 6ms/step - loss: 1.2380 - accuracy: 0.4656 - val_loss: 1.4281 - val_accuracy: 0.3636
Epoch 5/100
6/6 [==============================] - 0s 6ms/step - loss: 1.1624 - accuracy: 0.5420 - val_loss: 1.4006 - val_accuracy: 0.4091
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1027 - accuracy: 0.6031 - val_loss: 1.3721 - val_accuracy: 0.4091
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0293 - accuracy: 0.6450 - val_loss: 1.3450 - val_accuracy: 0.4242
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9552 - accuracy: 0.6679 - val_loss: 1.3221 - val_accuracy: 0.4545
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8725 - accuracy: 0.7405 - val_loss: 1.3101 - val_accuracy: 0.4394
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8125 - accuracy: 0.7748 - val_loss: 1.3102 - val_accuracy: 0.4697
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7457 - accuracy: 0.7863 - val_loss: 1.3186 - val_accuracy: 0.4545
Epoch 12/100
6/6 [==============================] - 0s 6ms/step - loss: 0.6867 - accuracy: 0.8130 - val_loss: 1.3286 - val_accuracy: 0.4242
11/11 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_15"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_45 (Dense)            (None, 150)               33000     
                                                                 
 dropout_15 (Dropout)        (None, 150)               0         
                                                                 
 dense_46 (Dense)            (None, 50)                7550      
                                                                 
 dense_47 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 37ms/step - loss: 1.6026 - accuracy: 0.2176 - val_loss: 1.5337 - val_accuracy: 0.3485
Epoch 2/100
6/6 [==============================] - 0s 8ms/step - loss: 1.4399 - accuracy: 0.3969 - val_loss: 1.4890 - val_accuracy: 0.3485
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3561 - accuracy: 0.4122 - val_loss: 1.4727 - val_accuracy: 0.3788
Epoch 4/100
6/6 [==============================] - 0s 10ms/step - loss: 1.2755 - accuracy: 0.4580 - val_loss: 1.4621 - val_accuracy: 0.3485
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2004 - accuracy: 0.5458 - val_loss: 1.4409 - val_accuracy: 0.3636
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1313 - accuracy: 0.5954 - val_loss: 1.4167 - val_accuracy: 0.3788
Epoch 7/100
6/6 [==============================] - 0s 6ms/step - loss: 1.0540 - accuracy: 0.6641 - val_loss: 1.3942 - val_accuracy: 0.3788
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9938 - accuracy: 0.6870 - val_loss: 1.3698 - val_accuracy: 0.3939
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8888 - accuracy: 0.7290 - val_loss: 1.3500 - val_accuracy: 0.4091
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8419 - accuracy: 0.7634 - val_loss: 1.3452 - val_accuracy: 0.4242
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7742 - accuracy: 0.8053 - val_loss: 1.3506 - val_accuracy: 0.4242
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 0.6863 - accuracy: 0.8282 - val_loss: 1.3561 - val_accuracy: 0.4242
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 0.6219 - accuracy: 0.8359 - val_loss: 1.3681 - val_accuracy: 0.4091
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_16"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_48 (Dense)            (None, 150)               33000     
                                                                 
 dropout_16 (Dropout)        (None, 150)               0         
                                                                 
 dense_49 (Dense)            (None, 50)                7550      
                                                                 
 dense_50 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 38ms/step - loss: 1.6931 - accuracy: 0.1832 - val_loss: 1.5735 - val_accuracy: 0.2273
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5018 - accuracy: 0.3473 - val_loss: 1.5097 - val_accuracy: 0.2879
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3756 - accuracy: 0.5000 - val_loss: 1.4824 - val_accuracy: 0.3030
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3003 - accuracy: 0.5153 - val_loss: 1.4680 - val_accuracy: 0.3485
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2260 - accuracy: 0.5496 - val_loss: 1.4553 - val_accuracy: 0.3788
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1341 - accuracy: 0.6221 - val_loss: 1.4428 - val_accuracy: 0.3636
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0661 - accuracy: 0.6756 - val_loss: 1.4294 - val_accuracy: 0.3636
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9941 - accuracy: 0.6985 - val_loss: 1.4138 - val_accuracy: 0.3939
Epoch 9/100
6/6 [==============================] - 0s 9ms/step - loss: 0.9090 - accuracy: 0.7176 - val_loss: 1.3972 - val_accuracy: 0.3333
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8513 - accuracy: 0.7366 - val_loss: 1.3936 - val_accuracy: 0.3485
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7636 - accuracy: 0.7672 - val_loss: 1.3995 - val_accuracy: 0.3788
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 0.6831 - accuracy: 0.8053 - val_loss: 1.3821 - val_accuracy: 0.3485
Epoch 13/100
6/6 [==============================] - 0s 10ms/step - loss: 0.6102 - accuracy: 0.8550 - val_loss: 1.3981 - val_accuracy: 0.3636
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 0.5761 - accuracy: 0.8321 - val_loss: 1.4198 - val_accuracy: 0.3788
Epoch 15/100
6/6 [==============================] - 0s 7ms/step - loss: 0.5016 - accuracy: 0.8664 - val_loss: 1.4422 - val_accuracy: 0.3939
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_17"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_51 (Dense)            (None, 150)               33000     
                                                                 
 dropout_17 (Dropout)        (None, 150)               0         
                                                                 
 dense_52 (Dense)            (None, 50)                7550      
                                                                 
 dense_53 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 38ms/step - loss: 1.6607 - accuracy: 0.1947 - val_loss: 1.5219 - val_accuracy: 0.3030
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4552 - accuracy: 0.3702 - val_loss: 1.4717 - val_accuracy: 0.3636
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3579 - accuracy: 0.4351 - val_loss: 1.4393 - val_accuracy: 0.3636
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2673 - accuracy: 0.4885 - val_loss: 1.4198 - val_accuracy: 0.3939
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1807 - accuracy: 0.5916 - val_loss: 1.4016 - val_accuracy: 0.4091
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1363 - accuracy: 0.5802 - val_loss: 1.3864 - val_accuracy: 0.3939
Epoch 7/100
6/6 [==============================] - 0s 10ms/step - loss: 1.0265 - accuracy: 0.6489 - val_loss: 1.3695 - val_accuracy: 0.4091
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9867 - accuracy: 0.6794 - val_loss: 1.3509 - val_accuracy: 0.4545
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9070 - accuracy: 0.7061 - val_loss: 1.3363 - val_accuracy: 0.4242
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8367 - accuracy: 0.7710 - val_loss: 1.3323 - val_accuracy: 0.4242
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7445 - accuracy: 0.7901 - val_loss: 1.3331 - val_accuracy: 0.4242
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 0.6882 - accuracy: 0.8282 - val_loss: 1.3280 - val_accuracy: 0.4545
Epoch 13/100
6/6 [==============================] - 0s 12ms/step - loss: 0.6386 - accuracy: 0.8053 - val_loss: 1.3386 - val_accuracy: 0.4545
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 0.5843 - accuracy: 0.8321 - val_loss: 1.3621 - val_accuracy: 0.4545
Epoch 15/100
6/6 [==============================] - 0s 7ms/step - loss: 0.5014 - accuracy: 0.8550 - val_loss: 1.3758 - val_accuracy: 0.4545
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_18"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_54 (Dense)            (None, 150)               33000     
                                                                 
 dropout_18 (Dropout)        (None, 150)               0         
                                                                 
 dense_55 (Dense)            (None, 50)                7550      
                                                                 
 dense_56 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 38ms/step - loss: 1.5647 - accuracy: 0.2939 - val_loss: 1.5168 - val_accuracy: 0.2879
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4327 - accuracy: 0.3702 - val_loss: 1.4814 - val_accuracy: 0.3636
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3355 - accuracy: 0.4237 - val_loss: 1.4485 - val_accuracy: 0.3485
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2634 - accuracy: 0.4847 - val_loss: 1.4207 - val_accuracy: 0.3485
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1788 - accuracy: 0.5916 - val_loss: 1.3941 - val_accuracy: 0.3636
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1141 - accuracy: 0.6412 - val_loss: 1.3693 - val_accuracy: 0.3636
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0417 - accuracy: 0.6985 - val_loss: 1.3420 - val_accuracy: 0.3939
Epoch 8/100
6/6 [==============================] - 0s 8ms/step - loss: 0.9768 - accuracy: 0.7405 - val_loss: 1.3128 - val_accuracy: 0.4091
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8962 - accuracy: 0.7443 - val_loss: 1.2939 - val_accuracy: 0.4242
Epoch 10/100
6/6 [==============================] - 0s 10ms/step - loss: 0.8209 - accuracy: 0.7977 - val_loss: 1.2852 - val_accuracy: 0.4091
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7499 - accuracy: 0.8244 - val_loss: 1.2819 - val_accuracy: 0.3939
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 0.6759 - accuracy: 0.8435 - val_loss: 1.2677 - val_accuracy: 0.4848
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 0.6406 - accuracy: 0.8397 - val_loss: 1.2675 - val_accuracy: 0.5000
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 0.5625 - accuracy: 0.8588 - val_loss: 1.2836 - val_accuracy: 0.4242
Epoch 15/100
6/6 [==============================] - 0s 7ms/step - loss: 0.4916 - accuracy: 0.9008 - val_loss: 1.3069 - val_accuracy: 0.4242
Epoch 16/100
6/6 [==============================] - 0s 7ms/step - loss: 0.4273 - accuracy: 0.9351 - val_loss: 1.3347 - val_accuracy: 0.4545
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_19"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_57 (Dense)            (None, 150)               33000     
                                                                 
 dropout_19 (Dropout)        (None, 150)               0         
                                                                 
 dense_58 (Dense)            (None, 50)                7550      
                                                                 
 dense_59 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 52ms/step - loss: 1.6342 - accuracy: 0.2405 - val_loss: 1.5711 - val_accuracy: 0.2121
Epoch 2/100
6/6 [==============================] - 0s 10ms/step - loss: 1.4670 - accuracy: 0.3626 - val_loss: 1.5290 - val_accuracy: 0.1970
Epoch 3/100
6/6 [==============================] - 0s 11ms/step - loss: 1.3690 - accuracy: 0.4122 - val_loss: 1.5019 - val_accuracy: 0.2424
Epoch 4/100
6/6 [==============================] - 0s 10ms/step - loss: 1.2932 - accuracy: 0.4466 - val_loss: 1.4848 - val_accuracy: 0.2879
Epoch 5/100
6/6 [==============================] - 0s 10ms/step - loss: 1.2306 - accuracy: 0.5191 - val_loss: 1.4645 - val_accuracy: 0.3182
Epoch 6/100
6/6 [==============================] - 0s 10ms/step - loss: 1.1655 - accuracy: 0.5534 - val_loss: 1.4421 - val_accuracy: 0.3030
Epoch 7/100
6/6 [==============================] - 0s 11ms/step - loss: 1.0974 - accuracy: 0.6069 - val_loss: 1.4239 - val_accuracy: 0.3030
Epoch 8/100
6/6 [==============================] - 0s 13ms/step - loss: 1.0226 - accuracy: 0.6412 - val_loss: 1.4124 - val_accuracy: 0.3182
Epoch 9/100
6/6 [==============================] - 0s 10ms/step - loss: 0.9760 - accuracy: 0.6603 - val_loss: 1.4041 - val_accuracy: 0.3485
Epoch 10/100
6/6 [==============================] - 0s 10ms/step - loss: 0.8954 - accuracy: 0.7176 - val_loss: 1.4054 - val_accuracy: 0.3182
Epoch 11/100
6/6 [==============================] - 0s 10ms/step - loss: 0.8406 - accuracy: 0.7443 - val_loss: 1.4177 - val_accuracy: 0.3030
Epoch 12/100
6/6 [==============================] - 0s 11ms/step - loss: 0.7638 - accuracy: 0.7710 - val_loss: 1.3984 - val_accuracy: 0.2879
Epoch 13/100
6/6 [==============================] - 0s 10ms/step - loss: 0.6919 - accuracy: 0.8282 - val_loss: 1.4067 - val_accuracy: 0.3636
Epoch 14/100
6/6 [==============================] - 0s 10ms/step - loss: 0.6447 - accuracy: 0.8130 - val_loss: 1.4301 - val_accuracy: 0.3333
Epoch 15/100
6/6 [==============================] - 0s 10ms/step - loss: 0.5877 - accuracy: 0.8435 - val_loss: 1.4649 - val_accuracy: 0.3182
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_20"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_60 (Dense)            (None, 150)               33000     
                                                                 
 dropout_20 (Dropout)        (None, 150)               0         
                                                                 
 dense_61 (Dense)            (None, 50)                7550      
                                                                 
 dense_62 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 39ms/step - loss: 1.5755 - accuracy: 0.3206 - val_loss: 1.5178 - val_accuracy: 0.3182
Epoch 2/100
6/6 [==============================] - 0s 8ms/step - loss: 1.4321 - accuracy: 0.3817 - val_loss: 1.4874 - val_accuracy: 0.3182
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3584 - accuracy: 0.4046 - val_loss: 1.4726 - val_accuracy: 0.3030
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2899 - accuracy: 0.5076 - val_loss: 1.4642 - val_accuracy: 0.3030
Epoch 5/100
6/6 [==============================] - 0s 9ms/step - loss: 1.2113 - accuracy: 0.5534 - val_loss: 1.4532 - val_accuracy: 0.3182
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1616 - accuracy: 0.5573 - val_loss: 1.4402 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 8ms/step - loss: 1.0785 - accuracy: 0.5992 - val_loss: 1.4232 - val_accuracy: 0.3636
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0061 - accuracy: 0.6947 - val_loss: 1.4011 - val_accuracy: 0.3939
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9313 - accuracy: 0.7137 - val_loss: 1.3769 - val_accuracy: 0.4091
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8645 - accuracy: 0.7557 - val_loss: 1.3609 - val_accuracy: 0.4242
Epoch 11/100
6/6 [==============================] - 0s 8ms/step - loss: 0.7898 - accuracy: 0.8130 - val_loss: 1.3477 - val_accuracy: 0.4242
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7170 - accuracy: 0.8015 - val_loss: 1.3348 - val_accuracy: 0.4091
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 0.6665 - accuracy: 0.8473 - val_loss: 1.3423 - val_accuracy: 0.4545
Epoch 14/100
6/6 [==============================] - 0s 8ms/step - loss: 0.5699 - accuracy: 0.8779 - val_loss: 1.3615 - val_accuracy: 0.4394
Epoch 15/100
6/6 [==============================] - 0s 8ms/step - loss: 0.5125 - accuracy: 0.8702 - val_loss: 1.3754 - val_accuracy: 0.4697
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_21"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_63 (Dense)            (None, 150)               33000     
                                                                 
 dropout_21 (Dropout)        (None, 150)               0         
                                                                 
 dense_64 (Dense)            (None, 50)                7550      
                                                                 
 dense_65 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 42ms/step - loss: 1.5639 - accuracy: 0.2939 - val_loss: 1.5179 - val_accuracy: 0.3030
Epoch 2/100
6/6 [==============================] - 0s 8ms/step - loss: 1.4074 - accuracy: 0.4008 - val_loss: 1.4873 - val_accuracy: 0.3030
Epoch 3/100
6/6 [==============================] - 0s 11ms/step - loss: 1.3366 - accuracy: 0.4466 - val_loss: 1.4682 - val_accuracy: 0.2727
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2661 - accuracy: 0.5191 - val_loss: 1.4549 - val_accuracy: 0.3030
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1810 - accuracy: 0.5649 - val_loss: 1.4344 - val_accuracy: 0.3182
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0945 - accuracy: 0.6183 - val_loss: 1.4073 - val_accuracy: 0.3182
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0462 - accuracy: 0.6145 - val_loss: 1.3801 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 8ms/step - loss: 0.9683 - accuracy: 0.6641 - val_loss: 1.3578 - val_accuracy: 0.3333
Epoch 9/100
6/6 [==============================] - 0s 8ms/step - loss: 0.8803 - accuracy: 0.7634 - val_loss: 1.3371 - val_accuracy: 0.3333
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8396 - accuracy: 0.7634 - val_loss: 1.3357 - val_accuracy: 0.4091
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7443 - accuracy: 0.8053 - val_loss: 1.3394 - val_accuracy: 0.3788
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 0.6883 - accuracy: 0.8206 - val_loss: 1.3153 - val_accuracy: 0.3939
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 0.6199 - accuracy: 0.8397 - val_loss: 1.3236 - val_accuracy: 0.3788
Epoch 14/100
6/6 [==============================] - 0s 8ms/step - loss: 0.5724 - accuracy: 0.8550 - val_loss: 1.3601 - val_accuracy: 0.3788
Epoch 15/100
6/6 [==============================] - 0s 11ms/step - loss: 0.5049 - accuracy: 0.8969 - val_loss: 1.3919 - val_accuracy: 0.3788
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step

NN_Model(X_train_cvfull_smote, X_test_cvfull, y_train_cvfull_smote, y_test_cvfull)

Model: "sequential_77"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_231 (Dense)           (None, 150)               33000     
                                                                 
 dropout_77 (Dropout)        (None, 150)               0         
                                                                 
 dense_232 (Dense)           (None, 50)                7550      
                                                                 
 dense_233 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 63ms/step - loss: 1.5764 - accuracy: 0.2227 - val_loss: 1.7461 - val_accuracy: 0.1364
Epoch 2/100
9/9 [==============================] - 0s 13ms/step - loss: 1.4058 - accuracy: 0.4568 - val_loss: 1.9086 - val_accuracy: 0.0545
Epoch 3/100
9/9 [==============================] - 0s 13ms/step - loss: 1.2793 - accuracy: 0.5386 - val_loss: 1.9666 - val_accuracy: 0.0636
Epoch 4/100
9/9 [==============================] - 0s 14ms/step - loss: 1.1688 - accuracy: 0.6227 - val_loss: 1.9702 - val_accuracy: 0.0636
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.56	0.39759	0.496869	0.356251

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(10):
  result=NN_Model(X_train_cvfull_smote, X_test_cvfull, y_train_cvfull_smote, y_test_cvfull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('F1 score')
plt.ylabel('F1 score')
plt.xlabel('epoch')
plt.show()

Model: "sequential_78"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_234 (Dense)           (None, 150)               33000     
                                                                 
 dropout_78 (Dropout)        (None, 150)               0         
                                                                 
 dense_235 (Dense)           (None, 50)                7550      
                                                                 
 dense_236 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 63ms/step - loss: 1.5800 - accuracy: 0.2932 - val_loss: 1.7875 - val_accuracy: 0.0818
Epoch 2/100
9/9 [==============================] - 0s 17ms/step - loss: 1.3932 - accuracy: 0.4841 - val_loss: 1.9720 - val_accuracy: 0.0545
Epoch 3/100
9/9 [==============================] - 0s 14ms/step - loss: 1.2604 - accuracy: 0.5795 - val_loss: 1.9972 - val_accuracy: 0.0455
Epoch 4/100
9/9 [==============================] - 0s 13ms/step - loss: 1.1321 - accuracy: 0.6386 - val_loss: 1.9554 - val_accuracy: 0.0545
18/18 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 4ms/step
18/18 [==============================] - 0s 4ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_79"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_237 (Dense)           (None, 150)               33000     
                                                                 
 dropout_79 (Dropout)        (None, 150)               0         
                                                                 
 dense_238 (Dense)           (None, 50)                7550      
                                                                 
 dense_239 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 54ms/step - loss: 1.5211 - accuracy: 0.3432 - val_loss: 1.8903 - val_accuracy: 0.0091
Epoch 2/100
9/9 [==============================] - 0s 10ms/step - loss: 1.3593 - accuracy: 0.4818 - val_loss: 2.0280 - val_accuracy: 0.0091
Epoch 3/100
9/9 [==============================] - 0s 10ms/step - loss: 1.2120 - accuracy: 0.5818 - val_loss: 2.0508 - val_accuracy: 0.0182
Epoch 4/100
9/9 [==============================] - 0s 9ms/step - loss: 1.1016 - accuracy: 0.6523 - val_loss: 2.0389 - val_accuracy: 0.0273
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_80"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_240 (Dense)           (None, 150)               33000     
                                                                 
 dropout_80 (Dropout)        (None, 150)               0         
                                                                 
 dense_241 (Dense)           (None, 50)                7550      
                                                                 
 dense_242 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 24ms/step - loss: 1.5996 - accuracy: 0.2705 - val_loss: 1.7131 - val_accuracy: 0.1000
Epoch 2/100
9/9 [==============================] - 0s 6ms/step - loss: 1.4238 - accuracy: 0.4636 - val_loss: 1.8405 - val_accuracy: 0.0545
Epoch 3/100
9/9 [==============================] - 0s 7ms/step - loss: 1.2890 - accuracy: 0.5432 - val_loss: 1.9127 - val_accuracy: 0.0545
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.1550 - accuracy: 0.6000 - val_loss: 1.9387 - val_accuracy: 0.0727
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 2ms/step
Model: "sequential_81"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_243 (Dense)           (None, 150)               33000     
                                                                 
 dropout_81 (Dropout)        (None, 150)               0         
                                                                 
 dense_244 (Dense)           (None, 50)                7550      
                                                                 
 dense_245 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 28ms/step - loss: 1.5766 - accuracy: 0.2977 - val_loss: 1.8008 - val_accuracy: 0.0545
Epoch 2/100
9/9 [==============================] - 0s 6ms/step - loss: 1.4123 - accuracy: 0.4227 - val_loss: 1.9500 - val_accuracy: 0.0091
Epoch 3/100
9/9 [==============================] - 0s 8ms/step - loss: 1.2869 - accuracy: 0.5136 - val_loss: 1.9613 - val_accuracy: 0.0727
Epoch 4/100
9/9 [==============================] - 0s 6ms/step - loss: 1.1531 - accuracy: 0.6205 - val_loss: 1.9358 - val_accuracy: 0.0818
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_82"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_246 (Dense)           (None, 150)               33000     
                                                                 
 dropout_82 (Dropout)        (None, 150)               0         
                                                                 
 dense_247 (Dense)           (None, 50)                7550      
                                                                 
 dense_248 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 27ms/step - loss: 1.5916 - accuracy: 0.2273 - val_loss: 1.7626 - val_accuracy: 0.0091
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.4121 - accuracy: 0.4636 - val_loss: 1.9218 - val_accuracy: 0.0091
Epoch 3/100
9/9 [==============================] - 0s 5ms/step - loss: 1.2629 - accuracy: 0.5500 - val_loss: 1.9561 - val_accuracy: 0.0182
Epoch 4/100
9/9 [==============================] - 0s 7ms/step - loss: 1.1132 - accuracy: 0.6818 - val_loss: 1.9362 - val_accuracy: 0.0273
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_83"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_249 (Dense)           (None, 150)               33000     
                                                                 
 dropout_83 (Dropout)        (None, 150)               0         
                                                                 
 dense_250 (Dense)           (None, 50)                7550      
                                                                 
 dense_251 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 27ms/step - loss: 1.6910 - accuracy: 0.2000 - val_loss: 1.5611 - val_accuracy: 0.1727
Epoch 2/100
9/9 [==============================] - 0s 8ms/step - loss: 1.4932 - accuracy: 0.3773 - val_loss: 1.6419 - val_accuracy: 0.1000
Epoch 3/100
9/9 [==============================] - 0s 6ms/step - loss: 1.3582 - accuracy: 0.4909 - val_loss: 1.6622 - val_accuracy: 0.0364
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.2216 - accuracy: 0.6273 - val_loss: 1.7233 - val_accuracy: 0.0364
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 4ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_84"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_252 (Dense)           (None, 150)               33000     
                                                                 
 dropout_84 (Dropout)        (None, 150)               0         
                                                                 
 dense_253 (Dense)           (None, 50)                7550      
                                                                 
 dense_254 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 34ms/step - loss: 1.5840 - accuracy: 0.2477 - val_loss: 1.8025 - val_accuracy: 0.0273
Epoch 2/100
9/9 [==============================] - 0s 11ms/step - loss: 1.4293 - accuracy: 0.3886 - val_loss: 1.9752 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 8ms/step - loss: 1.3087 - accuracy: 0.4818 - val_loss: 1.9669 - val_accuracy: 0.0182
Epoch 4/100
9/9 [==============================] - 0s 9ms/step - loss: 1.1795 - accuracy: 0.6250 - val_loss: 1.8969 - val_accuracy: 0.0727
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 5ms/step
Model: "sequential_85"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_255 (Dense)           (None, 150)               33000     
                                                                 
 dropout_85 (Dropout)        (None, 150)               0         
                                                                 
 dense_256 (Dense)           (None, 50)                7550      
                                                                 
 dense_257 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 40ms/step - loss: 1.6104 - accuracy: 0.2500 - val_loss: 1.7527 - val_accuracy: 0.0818
Epoch 2/100
9/9 [==============================] - 0s 8ms/step - loss: 1.4475 - accuracy: 0.4023 - val_loss: 1.9177 - val_accuracy: 0.0182
Epoch 3/100
9/9 [==============================] - 0s 9ms/step - loss: 1.3287 - accuracy: 0.4455 - val_loss: 1.9836 - val_accuracy: 0.0182
Epoch 4/100
9/9 [==============================] - 0s 10ms/step - loss: 1.2119 - accuracy: 0.5477 - val_loss: 1.9833 - val_accuracy: 0.0273
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 5ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_86"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_258 (Dense)           (None, 150)               33000     
                                                                 
 dropout_86 (Dropout)        (None, 150)               0         
                                                                 
 dense_259 (Dense)           (None, 50)                7550      
                                                                 
 dense_260 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 27ms/step - loss: 1.5743 - accuracy: 0.2773 - val_loss: 1.7984 - val_accuracy: 0.0364
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3874 - accuracy: 0.4682 - val_loss: 1.9582 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 7ms/step - loss: 1.2570 - accuracy: 0.5659 - val_loss: 1.9835 - val_accuracy: 0.0091
Epoch 4/100
9/9 [==============================] - 0s 6ms/step - loss: 1.1289 - accuracy: 0.6568 - val_loss: 1.9355 - val_accuracy: 0.0364
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_87"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_261 (Dense)           (None, 150)               33000     
                                                                 
 dropout_87 (Dropout)        (None, 150)               0         
                                                                 
 dense_262 (Dense)           (None, 50)                7550      
                                                                 
 dense_263 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 25ms/step - loss: 1.6016 - accuracy: 0.2023 - val_loss: 1.6981 - val_accuracy: 0.1091
Epoch 2/100
9/9 [==============================] - 0s 8ms/step - loss: 1.4270 - accuracy: 0.4614 - val_loss: 1.8824 - val_accuracy: 0.0545
Epoch 3/100
9/9 [==============================] - 0s 7ms/step - loss: 1.3001 - accuracy: 0.5568 - val_loss: 1.9443 - val_accuracy: 0.0455
Epoch 4/100
9/9 [==============================] - 0s 8ms/step - loss: 1.1697 - accuracy: 0.6205 - val_loss: 1.9218 - val_accuracy: 0.0636
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step

Applying ANN function on TFIDF dataset-

NN_Model(X_train_tfidf, X_test_tfidf, y_train_tfidf, y_test_tfidf)

Model: "sequential_22"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_66 (Dense)            (None, 150)               30150     
                                                                 
 dropout_22 (Dropout)        (None, 150)               0         
                                                                 
 dense_67 (Dense)            (None, 50)                7550      
                                                                 
 dense_68 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 2s 100ms/step - loss: 1.6046 - accuracy: 0.2366 - val_loss: 1.5826 - val_accuracy: 0.2727
Epoch 2/100
6/6 [==============================] - 0s 22ms/step - loss: 1.5565 - accuracy: 0.3244 - val_loss: 1.5629 - val_accuracy: 0.2576
Epoch 3/100
6/6 [==============================] - 0s 22ms/step - loss: 1.5211 - accuracy: 0.4122 - val_loss: 1.5456 - val_accuracy: 0.2879
Epoch 4/100
6/6 [==============================] - 0s 21ms/step - loss: 1.4781 - accuracy: 0.4542 - val_loss: 1.5302 - val_accuracy: 0.3485
Epoch 5/100
6/6 [==============================] - 0s 27ms/step - loss: 1.4378 - accuracy: 0.5153 - val_loss: 1.5165 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 21ms/step - loss: 1.3916 - accuracy: 0.5344 - val_loss: 1.5076 - val_accuracy: 0.2879
Epoch 7/100
6/6 [==============================] - 0s 16ms/step - loss: 1.3461 - accuracy: 0.4695 - val_loss: 1.5052 - val_accuracy: 0.2879
Epoch 8/100
6/6 [==============================] - 0s 23ms/step - loss: 1.3128 - accuracy: 0.4962 - val_loss: 1.5058 - val_accuracy: 0.3030
Epoch 9/100
6/6 [==============================] - 0s 12ms/step - loss: 1.2686 - accuracy: 0.4962 - val_loss: 1.5063 - val_accuracy: 0.3030
Epoch 10/100
6/6 [==============================] - 0s 17ms/step - loss: 1.2300 - accuracy: 0.5305 - val_loss: 1.5071 - val_accuracy: 0.3030
11/11 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.539634	0.373494	0.475684	0.26126

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(10):
  result=NN_Model(X_train_tfidf, X_test_tfidf, y_train_tfidf, y_test_tfidf)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('F1 score')
plt.ylabel('F1 score')
plt.xlabel('epoch')
plt.show()

Model: "sequential_23"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_69 (Dense)            (None, 150)               30150     
                                                                 
 dropout_23 (Dropout)        (None, 150)               0         
                                                                 
 dense_70 (Dense)            (None, 50)                7550      
                                                                 
 dense_71 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 78ms/step - loss: 1.5936 - accuracy: 0.2328 - val_loss: 1.5740 - val_accuracy: 0.1970
Epoch 2/100
6/6 [==============================] - 0s 17ms/step - loss: 1.5401 - accuracy: 0.3092 - val_loss: 1.5487 - val_accuracy: 0.2576
Epoch 3/100
6/6 [==============================] - 0s 14ms/step - loss: 1.5009 - accuracy: 0.3626 - val_loss: 1.5265 - val_accuracy: 0.3485
Epoch 4/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4510 - accuracy: 0.4351 - val_loss: 1.5107 - val_accuracy: 0.3030
Epoch 5/100
6/6 [==============================] - 0s 11ms/step - loss: 1.4065 - accuracy: 0.5153 - val_loss: 1.5005 - val_accuracy: 0.3182
Epoch 6/100
6/6 [==============================] - 0s 11ms/step - loss: 1.3731 - accuracy: 0.4695 - val_loss: 1.4963 - val_accuracy: 0.2879
Epoch 7/100
6/6 [==============================] - 0s 13ms/step - loss: 1.3412 - accuracy: 0.4771 - val_loss: 1.4944 - val_accuracy: 0.3030
Epoch 8/100
6/6 [==============================] - 0s 13ms/step - loss: 1.3089 - accuracy: 0.4771 - val_loss: 1.4887 - val_accuracy: 0.2879
Epoch 9/100
6/6 [==============================] - 0s 11ms/step - loss: 1.2760 - accuracy: 0.4580 - val_loss: 1.4767 - val_accuracy: 0.2879
Epoch 10/100
6/6 [==============================] - 0s 11ms/step - loss: 1.2384 - accuracy: 0.5267 - val_loss: 1.4660 - val_accuracy: 0.3030
Epoch 11/100
6/6 [==============================] - 0s 10ms/step - loss: 1.1910 - accuracy: 0.5992 - val_loss: 1.4583 - val_accuracy: 0.3788
Epoch 12/100
6/6 [==============================] - 0s 12ms/step - loss: 1.1487 - accuracy: 0.6603 - val_loss: 1.4409 - val_accuracy: 0.3333
Epoch 13/100
6/6 [==============================] - 0s 12ms/step - loss: 1.0941 - accuracy: 0.6641 - val_loss: 1.4308 - val_accuracy: 0.3030
Epoch 14/100
6/6 [==============================] - 0s 11ms/step - loss: 1.0275 - accuracy: 0.7061 - val_loss: 1.4275 - val_accuracy: 0.3636
Epoch 15/100
6/6 [==============================] - 0s 15ms/step - loss: 0.9721 - accuracy: 0.7214 - val_loss: 1.4241 - val_accuracy: 0.3636
Epoch 16/100
6/6 [==============================] - 0s 16ms/step - loss: 0.9054 - accuracy: 0.7595 - val_loss: 1.4278 - val_accuracy: 0.3939
Epoch 17/100
6/6 [==============================] - 0s 13ms/step - loss: 0.8498 - accuracy: 0.7634 - val_loss: 1.4169 - val_accuracy: 0.3636
Epoch 18/100
6/6 [==============================] - 0s 16ms/step - loss: 0.7764 - accuracy: 0.7901 - val_loss: 1.4234 - val_accuracy: 0.3636
Epoch 19/100
6/6 [==============================] - 0s 16ms/step - loss: 0.7170 - accuracy: 0.8244 - val_loss: 1.4355 - val_accuracy: 0.3788
Epoch 20/100
6/6 [==============================] - 0s 27ms/step - loss: 0.6585 - accuracy: 0.8397 - val_loss: 1.4336 - val_accuracy: 0.3333
11/11 [==============================] - 0s 5ms/step
3/3 [==============================] - 0s 8ms/step
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 5ms/step
Model: "sequential_24"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_72 (Dense)            (None, 150)               30150     
                                                                 
 dropout_24 (Dropout)        (None, 150)               0         
                                                                 
 dense_73 (Dense)            (None, 50)                7550      
                                                                 
 dense_74 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 2s 106ms/step - loss: 1.5973 - accuracy: 0.2099 - val_loss: 1.5913 - val_accuracy: 0.1970
Epoch 2/100
6/6 [==============================] - 0s 19ms/step - loss: 1.5538 - accuracy: 0.3550 - val_loss: 1.5706 - val_accuracy: 0.2121
Epoch 3/100
6/6 [==============================] - 0s 12ms/step - loss: 1.5135 - accuracy: 0.4924 - val_loss: 1.5519 - val_accuracy: 0.2121
Epoch 4/100
6/6 [==============================] - 0s 9ms/step - loss: 1.4722 - accuracy: 0.5382 - val_loss: 1.5346 - val_accuracy: 0.2121
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4300 - accuracy: 0.5534 - val_loss: 1.5206 - val_accuracy: 0.2424
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3801 - accuracy: 0.5763 - val_loss: 1.5131 - val_accuracy: 0.2576
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3440 - accuracy: 0.5382 - val_loss: 1.5100 - val_accuracy: 0.3485
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3105 - accuracy: 0.5458 - val_loss: 1.5070 - val_accuracy: 0.3636
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2642 - accuracy: 0.5687 - val_loss: 1.5002 - val_accuracy: 0.3333
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2227 - accuracy: 0.5687 - val_loss: 1.4953 - val_accuracy: 0.3333
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1813 - accuracy: 0.6031 - val_loss: 1.4931 - val_accuracy: 0.3485
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1248 - accuracy: 0.6870 - val_loss: 1.4747 - val_accuracy: 0.3182
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0692 - accuracy: 0.7214 - val_loss: 1.4640 - val_accuracy: 0.3485
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0135 - accuracy: 0.7137 - val_loss: 1.4635 - val_accuracy: 0.3636
Epoch 15/100
6/6 [==============================] - 0s 8ms/step - loss: 0.9554 - accuracy: 0.7519 - val_loss: 1.4652 - val_accuracy: 0.3333
Epoch 16/100
6/6 [==============================] - 0s 8ms/step - loss: 0.8870 - accuracy: 0.7634 - val_loss: 1.4828 - val_accuracy: 0.3333
Epoch 17/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8269 - accuracy: 0.7748 - val_loss: 1.4766 - val_accuracy: 0.3333
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_25"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_75 (Dense)            (None, 150)               30150     
                                                                 
 dropout_25 (Dropout)        (None, 150)               0         
                                                                 
 dense_76 (Dense)            (None, 50)                7550      
                                                                 
 dense_77 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 38ms/step - loss: 1.5940 - accuracy: 0.3244 - val_loss: 1.5753 - val_accuracy: 0.3182
Epoch 2/100
6/6 [==============================] - 0s 8ms/step - loss: 1.5472 - accuracy: 0.3359 - val_loss: 1.5528 - val_accuracy: 0.3333
Epoch 3/100
6/6 [==============================] - 0s 8ms/step - loss: 1.5075 - accuracy: 0.3473 - val_loss: 1.5327 - val_accuracy: 0.3333
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4642 - accuracy: 0.3473 - val_loss: 1.5164 - val_accuracy: 0.3333
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4180 - accuracy: 0.3550 - val_loss: 1.5052 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3798 - accuracy: 0.3664 - val_loss: 1.5019 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 11ms/step - loss: 1.3390 - accuracy: 0.3817 - val_loss: 1.5024 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3028 - accuracy: 0.4198 - val_loss: 1.5024 - val_accuracy: 0.3333
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2670 - accuracy: 0.4351 - val_loss: 1.4986 - val_accuracy: 0.3182
Epoch 10/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2225 - accuracy: 0.4924 - val_loss: 1.4934 - val_accuracy: 0.3333
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1867 - accuracy: 0.6069 - val_loss: 1.4961 - val_accuracy: 0.3182
Epoch 12/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1315 - accuracy: 0.6221 - val_loss: 1.4861 - val_accuracy: 0.3182
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0865 - accuracy: 0.6374 - val_loss: 1.4805 - val_accuracy: 0.3182
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0283 - accuracy: 0.6298 - val_loss: 1.4822 - val_accuracy: 0.3182
Epoch 15/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9778 - accuracy: 0.6870 - val_loss: 1.4839 - val_accuracy: 0.2879
Epoch 16/100
6/6 [==============================] - 0s 11ms/step - loss: 0.9046 - accuracy: 0.7443 - val_loss: 1.4938 - val_accuracy: 0.3333
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_26"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_78 (Dense)            (None, 150)               30150     
                                                                 
 dropout_26 (Dropout)        (None, 150)               0         
                                                                 
 dense_79 (Dense)            (None, 50)                7550      
                                                                 
 dense_80 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 41ms/step - loss: 1.5914 - accuracy: 0.2672 - val_loss: 1.5843 - val_accuracy: 0.2273
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5404 - accuracy: 0.3435 - val_loss: 1.5609 - val_accuracy: 0.2424
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5001 - accuracy: 0.3702 - val_loss: 1.5386 - val_accuracy: 0.2727
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4537 - accuracy: 0.3969 - val_loss: 1.5184 - val_accuracy: 0.2879
Epoch 5/100
6/6 [==============================] - 0s 10ms/step - loss: 1.4079 - accuracy: 0.3969 - val_loss: 1.5020 - val_accuracy: 0.3182
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3691 - accuracy: 0.3817 - val_loss: 1.4920 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3227 - accuracy: 0.3969 - val_loss: 1.4856 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2848 - accuracy: 0.4046 - val_loss: 1.4814 - val_accuracy: 0.3333
Epoch 9/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2418 - accuracy: 0.4160 - val_loss: 1.4727 - val_accuracy: 0.3333
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1958 - accuracy: 0.4847 - val_loss: 1.4637 - val_accuracy: 0.3788
Epoch 11/100
6/6 [==============================] - 0s 10ms/step - loss: 1.1617 - accuracy: 0.6107 - val_loss: 1.4601 - val_accuracy: 0.3788
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0951 - accuracy: 0.6794 - val_loss: 1.4433 - val_accuracy: 0.3636
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0296 - accuracy: 0.7137 - val_loss: 1.4322 - val_accuracy: 0.3485
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9816 - accuracy: 0.7214 - val_loss: 1.4273 - val_accuracy: 0.3485
Epoch 15/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9066 - accuracy: 0.7519 - val_loss: 1.4226 - val_accuracy: 0.3485
Epoch 16/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8560 - accuracy: 0.7748 - val_loss: 1.4351 - val_accuracy: 0.3485
Epoch 17/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7956 - accuracy: 0.8130 - val_loss: 1.4304 - val_accuracy: 0.3485
Epoch 18/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7322 - accuracy: 0.7939 - val_loss: 1.4416 - val_accuracy: 0.3485
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_27"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_81 (Dense)            (None, 150)               30150     
                                                                 
 dropout_27 (Dropout)        (None, 150)               0         
                                                                 
 dense_82 (Dense)            (None, 50)                7550      
                                                                 
 dense_83 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 37ms/step - loss: 1.6102 - accuracy: 0.2023 - val_loss: 1.5998 - val_accuracy: 0.1818
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5545 - accuracy: 0.4237 - val_loss: 1.5793 - val_accuracy: 0.2424
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5195 - accuracy: 0.4046 - val_loss: 1.5618 - val_accuracy: 0.2424
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4770 - accuracy: 0.4313 - val_loss: 1.5477 - val_accuracy: 0.2727
Epoch 5/100
6/6 [==============================] - 0s 8ms/step - loss: 1.4317 - accuracy: 0.4885 - val_loss: 1.5349 - val_accuracy: 0.2576
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3813 - accuracy: 0.5115 - val_loss: 1.5259 - val_accuracy: 0.2121
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3384 - accuracy: 0.4962 - val_loss: 1.5220 - val_accuracy: 0.2576
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3050 - accuracy: 0.5115 - val_loss: 1.5192 - val_accuracy: 0.2879
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2610 - accuracy: 0.5038 - val_loss: 1.5142 - val_accuracy: 0.2879
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2173 - accuracy: 0.5687 - val_loss: 1.5090 - val_accuracy: 0.3182
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1740 - accuracy: 0.6374 - val_loss: 1.5075 - val_accuracy: 0.2424
Epoch 12/100
6/6 [==============================] - 0s 9ms/step - loss: 1.1178 - accuracy: 0.6679 - val_loss: 1.4935 - val_accuracy: 0.3030
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0666 - accuracy: 0.7061 - val_loss: 1.4849 - val_accuracy: 0.3333
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9970 - accuracy: 0.7252 - val_loss: 1.4855 - val_accuracy: 0.3182
Epoch 15/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9302 - accuracy: 0.7824 - val_loss: 1.4895 - val_accuracy: 0.3182
Epoch 16/100
6/6 [==============================] - 0s 9ms/step - loss: 0.8704 - accuracy: 0.7557 - val_loss: 1.5059 - val_accuracy: 0.3030
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_28"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_84 (Dense)            (None, 150)               30150     
                                                                 
 dropout_28 (Dropout)        (None, 150)               0         
                                                                 
 dense_85 (Dense)            (None, 50)                7550      
                                                                 
 dense_86 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 39ms/step - loss: 1.5853 - accuracy: 0.3168 - val_loss: 1.5876 - val_accuracy: 0.2727
Epoch 2/100
6/6 [==============================] - 0s 8ms/step - loss: 1.5361 - accuracy: 0.3473 - val_loss: 1.5641 - val_accuracy: 0.3333
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4975 - accuracy: 0.3588 - val_loss: 1.5454 - val_accuracy: 0.3182
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4545 - accuracy: 0.3550 - val_loss: 1.5300 - val_accuracy: 0.3333
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4090 - accuracy: 0.3435 - val_loss: 1.5192 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3683 - accuracy: 0.3511 - val_loss: 1.5152 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3310 - accuracy: 0.3511 - val_loss: 1.5152 - val_accuracy: 0.3182
Epoch 8/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2921 - accuracy: 0.3855 - val_loss: 1.5167 - val_accuracy: 0.3182
Epoch 9/100
6/6 [==============================] - 0s 9ms/step - loss: 1.2485 - accuracy: 0.4580 - val_loss: 1.5159 - val_accuracy: 0.2879
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2114 - accuracy: 0.5344 - val_loss: 1.5120 - val_accuracy: 0.2727
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1626 - accuracy: 0.5916 - val_loss: 1.5094 - val_accuracy: 0.2879
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1067 - accuracy: 0.6489 - val_loss: 1.4927 - val_accuracy: 0.2879
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0523 - accuracy: 0.6603 - val_loss: 1.4819 - val_accuracy: 0.2879
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9994 - accuracy: 0.6985 - val_loss: 1.4788 - val_accuracy: 0.3030
Epoch 15/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9261 - accuracy: 0.7443 - val_loss: 1.4783 - val_accuracy: 0.3485
Epoch 16/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8644 - accuracy: 0.7863 - val_loss: 1.4900 - val_accuracy: 0.3485
Epoch 17/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8087 - accuracy: 0.7824 - val_loss: 1.4837 - val_accuracy: 0.3333
Epoch 18/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7226 - accuracy: 0.8092 - val_loss: 1.4993 - val_accuracy: 0.3182
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_29"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_87 (Dense)            (None, 150)               30150     
                                                                 
 dropout_29 (Dropout)        (None, 150)               0         
                                                                 
 dense_88 (Dense)            (None, 50)                7550      
                                                                 
 dense_89 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 61ms/step - loss: 1.5682 - accuracy: 0.2939 - val_loss: 1.5631 - val_accuracy: 0.2121
Epoch 2/100
6/6 [==============================] - 0s 11ms/step - loss: 1.5141 - accuracy: 0.3702 - val_loss: 1.5379 - val_accuracy: 0.2727
Epoch 3/100
6/6 [==============================] - 0s 10ms/step - loss: 1.4676 - accuracy: 0.3931 - val_loss: 1.5181 - val_accuracy: 0.3182
Epoch 4/100
6/6 [==============================] - 0s 10ms/step - loss: 1.4196 - accuracy: 0.3931 - val_loss: 1.5079 - val_accuracy: 0.3182
Epoch 5/100
6/6 [==============================] - 0s 10ms/step - loss: 1.3795 - accuracy: 0.3855 - val_loss: 1.5058 - val_accuracy: 0.3182
Epoch 6/100
6/6 [==============================] - 0s 10ms/step - loss: 1.3399 - accuracy: 0.3740 - val_loss: 1.5083 - val_accuracy: 0.3030
Epoch 7/100
6/6 [==============================] - 0s 10ms/step - loss: 1.3134 - accuracy: 0.3626 - val_loss: 1.5060 - val_accuracy: 0.3030
Epoch 8/100
6/6 [==============================] - 0s 10ms/step - loss: 1.2774 - accuracy: 0.3855 - val_loss: 1.4973 - val_accuracy: 0.3030
Epoch 9/100
6/6 [==============================] - 0s 9ms/step - loss: 1.2453 - accuracy: 0.4198 - val_loss: 1.4828 - val_accuracy: 0.2727
Epoch 10/100
6/6 [==============================] - 0s 10ms/step - loss: 1.1994 - accuracy: 0.5115 - val_loss: 1.4702 - val_accuracy: 0.3182
Epoch 11/100
6/6 [==============================] - 0s 11ms/step - loss: 1.1563 - accuracy: 0.6260 - val_loss: 1.4661 - val_accuracy: 0.3182
Epoch 12/100
6/6 [==============================] - 0s 9ms/step - loss: 1.1045 - accuracy: 0.6450 - val_loss: 1.4577 - val_accuracy: 0.3182
Epoch 13/100
6/6 [==============================] - 0s 9ms/step - loss: 1.0569 - accuracy: 0.6450 - val_loss: 1.4527 - val_accuracy: 0.3788
Epoch 14/100
6/6 [==============================] - 0s 14ms/step - loss: 0.9908 - accuracy: 0.7176 - val_loss: 1.4514 - val_accuracy: 0.3788
Epoch 15/100
6/6 [==============================] - 0s 10ms/step - loss: 0.9367 - accuracy: 0.7557 - val_loss: 1.4506 - val_accuracy: 0.3636
Epoch 16/100
6/6 [==============================] - 0s 9ms/step - loss: 0.8700 - accuracy: 0.7901 - val_loss: 1.4689 - val_accuracy: 0.3333
Epoch 17/100
6/6 [==============================] - 0s 9ms/step - loss: 0.8046 - accuracy: 0.8168 - val_loss: 1.4696 - val_accuracy: 0.3182
Epoch 18/100
6/6 [==============================] - 0s 12ms/step - loss: 0.7517 - accuracy: 0.8359 - val_loss: 1.4853 - val_accuracy: 0.3182
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step
Model: "sequential_30"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_90 (Dense)            (None, 150)               30150     
                                                                 
 dropout_30 (Dropout)        (None, 150)               0         
                                                                 
 dense_91 (Dense)            (None, 50)                7550      
                                                                 
 dense_92 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 37ms/step - loss: 1.6011 - accuracy: 0.2824 - val_loss: 1.5844 - val_accuracy: 0.2879
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5521 - accuracy: 0.3931 - val_loss: 1.5609 - val_accuracy: 0.3182
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5119 - accuracy: 0.3664 - val_loss: 1.5376 - val_accuracy: 0.3485
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4654 - accuracy: 0.3969 - val_loss: 1.5164 - val_accuracy: 0.3485
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4194 - accuracy: 0.3817 - val_loss: 1.5018 - val_accuracy: 0.3485
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3731 - accuracy: 0.4046 - val_loss: 1.4960 - val_accuracy: 0.3485
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3283 - accuracy: 0.3969 - val_loss: 1.4948 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2916 - accuracy: 0.4275 - val_loss: 1.4922 - val_accuracy: 0.3333
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2458 - accuracy: 0.4733 - val_loss: 1.4865 - val_accuracy: 0.3485
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2054 - accuracy: 0.5496 - val_loss: 1.4807 - val_accuracy: 0.3333
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1535 - accuracy: 0.6069 - val_loss: 1.4788 - val_accuracy: 0.3333
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0964 - accuracy: 0.6718 - val_loss: 1.4697 - val_accuracy: 0.3485
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0371 - accuracy: 0.7099 - val_loss: 1.4652 - val_accuracy: 0.3485
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9786 - accuracy: 0.7290 - val_loss: 1.4672 - val_accuracy: 0.3030
Epoch 15/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9213 - accuracy: 0.7672 - val_loss: 1.4690 - val_accuracy: 0.3333
Epoch 16/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8589 - accuracy: 0.7824 - val_loss: 1.4870 - val_accuracy: 0.3333
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_31"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_93 (Dense)            (None, 150)               30150     
                                                                 
 dropout_31 (Dropout)        (None, 150)               0         
                                                                 
 dense_94 (Dense)            (None, 50)                7550      
                                                                 
 dense_95 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 37ms/step - loss: 1.5600 - accuracy: 0.3397 - val_loss: 1.5588 - val_accuracy: 0.2879
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5067 - accuracy: 0.3817 - val_loss: 1.5354 - val_accuracy: 0.3030
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4574 - accuracy: 0.4275 - val_loss: 1.5186 - val_accuracy: 0.3182
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4162 - accuracy: 0.4504 - val_loss: 1.5080 - val_accuracy: 0.3333
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3732 - accuracy: 0.4351 - val_loss: 1.5033 - val_accuracy: 0.3636
Epoch 6/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3341 - accuracy: 0.4198 - val_loss: 1.5042 - val_accuracy: 0.3485
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3001 - accuracy: 0.4046 - val_loss: 1.5060 - val_accuracy: 0.3788
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2674 - accuracy: 0.4351 - val_loss: 1.5028 - val_accuracy: 0.3485
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2302 - accuracy: 0.4771 - val_loss: 1.4920 - val_accuracy: 0.3485
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1838 - accuracy: 0.5687 - val_loss: 1.4833 - val_accuracy: 0.3636
Epoch 11/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1395 - accuracy: 0.6336 - val_loss: 1.4769 - val_accuracy: 0.3333
Epoch 12/100
6/6 [==============================] - 0s 8ms/step - loss: 1.0864 - accuracy: 0.6794 - val_loss: 1.4685 - val_accuracy: 0.3485
Epoch 13/100
6/6 [==============================] - 0s 8ms/step - loss: 1.0258 - accuracy: 0.6870 - val_loss: 1.4664 - val_accuracy: 0.3939
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9800 - accuracy: 0.6832 - val_loss: 1.4691 - val_accuracy: 0.3939
Epoch 15/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9132 - accuracy: 0.7519 - val_loss: 1.4709 - val_accuracy: 0.3636
Epoch 16/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8741 - accuracy: 0.7672 - val_loss: 1.4795 - val_accuracy: 0.3333
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_32"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_96 (Dense)            (None, 150)               30150     
                                                                 
 dropout_32 (Dropout)        (None, 150)               0         
                                                                 
 dense_97 (Dense)            (None, 50)                7550      
                                                                 
 dense_98 (Dense)            (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 39ms/step - loss: 1.6074 - accuracy: 0.2099 - val_loss: 1.5930 - val_accuracy: 0.2121
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5545 - accuracy: 0.4160 - val_loss: 1.5734 - val_accuracy: 0.2273
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5175 - accuracy: 0.4084 - val_loss: 1.5552 - val_accuracy: 0.2424
Epoch 4/100
6/6 [==============================] - 0s 10ms/step - loss: 1.4795 - accuracy: 0.4389 - val_loss: 1.5405 - val_accuracy: 0.2424
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4347 - accuracy: 0.4733 - val_loss: 1.5250 - val_accuracy: 0.2879
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3949 - accuracy: 0.4771 - val_loss: 1.5131 - val_accuracy: 0.2879
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3439 - accuracy: 0.4695 - val_loss: 1.5050 - val_accuracy: 0.2879
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3108 - accuracy: 0.4733 - val_loss: 1.5002 - val_accuracy: 0.3030
Epoch 9/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2696 - accuracy: 0.4695 - val_loss: 1.4949 - val_accuracy: 0.3030
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2234 - accuracy: 0.5191 - val_loss: 1.4925 - val_accuracy: 0.3030
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1795 - accuracy: 0.5954 - val_loss: 1.4918 - val_accuracy: 0.2727
Epoch 12/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1187 - accuracy: 0.6527 - val_loss: 1.4760 - val_accuracy: 0.3182
Epoch 13/100
6/6 [==============================] - 0s 9ms/step - loss: 1.0614 - accuracy: 0.6908 - val_loss: 1.4646 - val_accuracy: 0.3030
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0093 - accuracy: 0.6985 - val_loss: 1.4653 - val_accuracy: 0.3030
Epoch 15/100
6/6 [==============================] - 0s 8ms/step - loss: 0.9463 - accuracy: 0.7328 - val_loss: 1.4651 - val_accuracy: 0.3030
Epoch 16/100
6/6 [==============================] - 0s 9ms/step - loss: 0.8942 - accuracy: 0.7557 - val_loss: 1.4777 - val_accuracy: 0.3182
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step

NN_Model(X_train_tfidf_smote, X_test_tfidf, y_train_tfidf_smote, y_test_tfidf)

Model: "sequential_88"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_264 (Dense)           (None, 150)               30150     
                                                                 
 dropout_88 (Dropout)        (None, 150)               0         
                                                                 
 dense_265 (Dense)           (None, 50)                7550      
                                                                 
 dense_266 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 49ms/step - loss: 1.5778 - accuracy: 0.3409 - val_loss: 1.7488 - val_accuracy: 0.0091
Epoch 2/100
9/9 [==============================] - 0s 10ms/step - loss: 1.5179 - accuracy: 0.4886 - val_loss: 1.9007 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 11ms/step - loss: 1.4432 - accuracy: 0.5432 - val_loss: 2.0563 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 11ms/step - loss: 1.3508 - accuracy: 0.5750 - val_loss: 2.1975 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.496364	0.337349	0.378326	0.282823

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(10):
  result=NN_Model(X_train_tfidf_smote, X_test_tfidf, y_train_tfidf_smote, y_test_tfidf)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('F1 score')
plt.ylabel('F1 score')
plt.xlabel('epoch')
plt.show()

Model: "sequential_142"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_426 (Dense)           (None, 150)               30150     
                                                                 
 dropout_142 (Dropout)       (None, 150)               0         
                                                                 
 dense_427 (Dense)           (None, 50)                7550      
                                                                 
 dense_428 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 25ms/step - loss: 1.5884 - accuracy: 0.2636 - val_loss: 1.7360 - val_accuracy: 0.0091
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.5347 - accuracy: 0.3705 - val_loss: 1.8711 - val_accuracy: 0.0091
Epoch 3/100
9/9 [==============================] - 0s 5ms/step - loss: 1.4751 - accuracy: 0.4886 - val_loss: 2.0157 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.4083 - accuracy: 0.5523 - val_loss: 2.1400 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_143"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_429 (Dense)           (None, 150)               30150     
                                                                 
 dropout_143 (Dropout)       (None, 150)               0         
                                                                 
 dense_430 (Dense)           (None, 50)                7550      
                                                                 
 dense_431 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 28ms/step - loss: 1.5920 - accuracy: 0.2568 - val_loss: 1.7067 - val_accuracy: 0.0182
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.5288 - accuracy: 0.4295 - val_loss: 1.8378 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 7ms/step - loss: 1.4594 - accuracy: 0.5409 - val_loss: 1.9553 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3750 - accuracy: 0.5795 - val_loss: 2.0598 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_144"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_432 (Dense)           (None, 150)               30150     
                                                                 
 dropout_144 (Dropout)       (None, 150)               0         
                                                                 
 dense_433 (Dense)           (None, 50)                7550      
                                                                 
 dense_434 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 28ms/step - loss: 1.5844 - accuracy: 0.2182 - val_loss: 1.7555 - val_accuracy: 0.0364
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.5234 - accuracy: 0.3386 - val_loss: 1.9096 - val_accuracy: 0.0273
Epoch 3/100
9/9 [==============================] - 0s 5ms/step - loss: 1.4581 - accuracy: 0.4636 - val_loss: 2.0564 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3831 - accuracy: 0.5818 - val_loss: 2.1813 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_145"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_435 (Dense)           (None, 150)               30150     
                                                                 
 dropout_145 (Dropout)       (None, 150)               0         
                                                                 
 dense_436 (Dense)           (None, 50)                7550      
                                                                 
 dense_437 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 25ms/step - loss: 1.5894 - accuracy: 0.2727 - val_loss: 1.6890 - val_accuracy: 0.0909
Epoch 2/100
9/9 [==============================] - 0s 6ms/step - loss: 1.5163 - accuracy: 0.4886 - val_loss: 1.8360 - val_accuracy: 0.0545
Epoch 3/100
9/9 [==============================] - 0s 6ms/step - loss: 1.4345 - accuracy: 0.5795 - val_loss: 1.9717 - val_accuracy: 0.0545
Epoch 4/100
9/9 [==============================] - 0s 8ms/step - loss: 1.3422 - accuracy: 0.6432 - val_loss: 2.0795 - val_accuracy: 0.0273
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_146"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_438 (Dense)           (None, 150)               30150     
                                                                 
 dropout_146 (Dropout)       (None, 150)               0         
                                                                 
 dense_439 (Dense)           (None, 50)                7550      
                                                                 
 dense_440 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 25ms/step - loss: 1.5918 - accuracy: 0.2295 - val_loss: 1.7084 - val_accuracy: 0.0182
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.5334 - accuracy: 0.4636 - val_loss: 1.8176 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 5ms/step - loss: 1.4680 - accuracy: 0.5818 - val_loss: 1.9453 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3887 - accuracy: 0.6000 - val_loss: 2.0846 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step
Model: "sequential_147"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_441 (Dense)           (None, 150)               30150     
                                                                 
 dropout_147 (Dropout)       (None, 150)               0         
                                                                 
 dense_442 (Dense)           (None, 50)                7550      
                                                                 
 dense_443 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 39ms/step - loss: 1.5884 - accuracy: 0.2682 - val_loss: 1.6999 - val_accuracy: 0.0364
Epoch 2/100
9/9 [==============================] - 0s 11ms/step - loss: 1.5335 - accuracy: 0.4864 - val_loss: 1.8297 - val_accuracy: 0.0091
Epoch 3/100
9/9 [==============================] - 0s 9ms/step - loss: 1.4761 - accuracy: 0.5500 - val_loss: 1.9438 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 8ms/step - loss: 1.4059 - accuracy: 0.6068 - val_loss: 2.0478 - val_accuracy: 0.0091
18/18 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 7ms/step
18/18 [==============================] - 0s 4ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_148"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_444 (Dense)           (None, 150)               30150     
                                                                 
 dropout_148 (Dropout)       (None, 150)               0         
                                                                 
 dense_445 (Dense)           (None, 50)                7550      
                                                                 
 dense_446 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 46ms/step - loss: 1.5922 - accuracy: 0.2591 - val_loss: 1.6938 - val_accuracy: 0.0545
Epoch 2/100
9/9 [==============================] - 0s 10ms/step - loss: 1.5377 - accuracy: 0.3568 - val_loss: 1.8031 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 9ms/step - loss: 1.4729 - accuracy: 0.4932 - val_loss: 1.9108 - val_accuracy: 0.0182
Epoch 4/100
9/9 [==============================] - 0s 9ms/step - loss: 1.3887 - accuracy: 0.6182 - val_loss: 2.0232 - val_accuracy: 0.0182
18/18 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_149"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_447 (Dense)           (None, 150)               30150     
                                                                 
 dropout_149 (Dropout)       (None, 150)               0         
                                                                 
 dense_448 (Dense)           (None, 50)                7550      
                                                                 
 dense_449 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 26ms/step - loss: 1.5796 - accuracy: 0.3227 - val_loss: 1.7051 - val_accuracy: 0.0636
Epoch 2/100
9/9 [==============================] - 0s 7ms/step - loss: 1.5064 - accuracy: 0.4364 - val_loss: 1.8210 - val_accuracy: 0.0091
Epoch 3/100
9/9 [==============================] - 0s 7ms/step - loss: 1.4217 - accuracy: 0.4886 - val_loss: 1.9350 - val_accuracy: 0.0091
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3160 - accuracy: 0.5068 - val_loss: 2.0506 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 2ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_150"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_450 (Dense)           (None, 150)               30150     
                                                                 
 dropout_150 (Dropout)       (None, 150)               0         
                                                                 
 dense_451 (Dense)           (None, 50)                7550      
                                                                 
 dense_452 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 28ms/step - loss: 1.5856 - accuracy: 0.3114 - val_loss: 1.7454 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.5179 - accuracy: 0.4773 - val_loss: 1.8801 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 7ms/step - loss: 1.4444 - accuracy: 0.5568 - val_loss: 2.0010 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3536 - accuracy: 0.5636 - val_loss: 2.1151 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_151"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_453 (Dense)           (None, 150)               30150     
                                                                 
 dropout_151 (Dropout)       (None, 150)               0         
                                                                 
 dense_454 (Dense)           (None, 50)                7550      
                                                                 
 dense_455 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 28ms/step - loss: 1.5758 - accuracy: 0.2568 - val_loss: 1.7954 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.5089 - accuracy: 0.3636 - val_loss: 1.9658 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 5ms/step - loss: 1.4329 - accuracy: 0.4682 - val_loss: 2.1028 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3508 - accuracy: 0.5727 - val_loss: 2.1878 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step

Applying ANN function on TFIDF full dataset-

NN_Model(X_train_tfidffull, X_test_tfidffull, y_train_tfidffull, y_test_tfidffull)

Model: "sequential_33"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_99 (Dense)            (None, 150)               33000     
                                                                 
 dropout_33 (Dropout)        (None, 150)               0         
                                                                 
 dense_100 (Dense)           (None, 50)                7550      
                                                                 
 dense_101 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 82ms/step - loss: 1.5732 - accuracy: 0.2748 - val_loss: 1.5362 - val_accuracy: 0.3030
Epoch 2/100
6/6 [==============================] - 0s 11ms/step - loss: 1.5008 - accuracy: 0.3359 - val_loss: 1.4888 - val_accuracy: 0.3485
Epoch 3/100
6/6 [==============================] - 0s 11ms/step - loss: 1.4341 - accuracy: 0.4008 - val_loss: 1.4578 - val_accuracy: 0.3485
Epoch 4/100
6/6 [==============================] - 0s 12ms/step - loss: 1.3788 - accuracy: 0.4389 - val_loss: 1.4365 - val_accuracy: 0.3636
Epoch 5/100
6/6 [==============================] - 0s 11ms/step - loss: 1.3178 - accuracy: 0.4809 - val_loss: 1.4253 - val_accuracy: 0.3939
Epoch 6/100
6/6 [==============================] - 0s 10ms/step - loss: 1.2756 - accuracy: 0.4695 - val_loss: 1.4198 - val_accuracy: 0.3939
Epoch 7/100
6/6 [==============================] - 0s 11ms/step - loss: 1.2431 - accuracy: 0.5382 - val_loss: 1.4067 - val_accuracy: 0.3788
Epoch 8/100
6/6 [==============================] - 0s 11ms/step - loss: 1.2083 - accuracy: 0.5534 - val_loss: 1.3905 - val_accuracy: 0.3636
Epoch 9/100
6/6 [==============================] - 0s 16ms/step - loss: 1.1636 - accuracy: 0.5763 - val_loss: 1.3701 - val_accuracy: 0.4091
Epoch 10/100
6/6 [==============================] - 0s 12ms/step - loss: 1.1256 - accuracy: 0.5916 - val_loss: 1.3537 - val_accuracy: 0.3788
Epoch 11/100
6/6 [==============================] - 0s 19ms/step - loss: 1.0954 - accuracy: 0.6298 - val_loss: 1.3401 - val_accuracy: 0.3939
Epoch 12/100
6/6 [==============================] - 0s 12ms/step - loss: 1.0326 - accuracy: 0.6260 - val_loss: 1.3119 - val_accuracy: 0.3939
Epoch 13/100
6/6 [==============================] - 0s 11ms/step - loss: 0.9820 - accuracy: 0.6641 - val_loss: 1.2986 - val_accuracy: 0.4242
Epoch 14/100
6/6 [==============================] - 0s 14ms/step - loss: 0.9216 - accuracy: 0.6870 - val_loss: 1.3006 - val_accuracy: 0.4091
Epoch 15/100
6/6 [==============================] - 0s 15ms/step - loss: 0.8773 - accuracy: 0.6985 - val_loss: 1.2985 - val_accuracy: 0.4091
Epoch 16/100
6/6 [==============================] - 0s 14ms/step - loss: 0.8076 - accuracy: 0.7595 - val_loss: 1.3003 - val_accuracy: 0.4091
Epoch 17/100
6/6 [==============================] - 0s 11ms/step - loss: 0.7808 - accuracy: 0.7405 - val_loss: 1.2875 - val_accuracy: 0.4242
Epoch 18/100
6/6 [==============================] - 0s 11ms/step - loss: 0.7131 - accuracy: 0.7634 - val_loss: 1.3163 - val_accuracy: 0.4242
Epoch 19/100
6/6 [==============================] - 0s 15ms/step - loss: 0.6741 - accuracy: 0.7634 - val_loss: 1.3231 - val_accuracy: 0.4394
Epoch 20/100
6/6 [==============================] - 0s 11ms/step - loss: 0.6220 - accuracy: 0.7939 - val_loss: 1.3054 - val_accuracy: 0.4545
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.771341	0.445783	0.756352	0.402492

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(10):
  result=NN_Model(X_train_tfidffull, X_test_tfidffull, y_train_tfidffull, y_test_tfidffull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('F1 score')
plt.ylabel('F1 score')
plt.xlabel('epoch')
plt.show()

Model: "sequential_89"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_267 (Dense)           (None, 150)               33000     
                                                                 
 dropout_89 (Dropout)        (None, 150)               0         
                                                                 
 dense_268 (Dense)           (None, 50)                7550      
                                                                 
 dense_269 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 38ms/step - loss: 1.5994 - accuracy: 0.2290 - val_loss: 1.5604 - val_accuracy: 0.3485
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5273 - accuracy: 0.3130 - val_loss: 1.5151 - val_accuracy: 0.3636
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4648 - accuracy: 0.4237 - val_loss: 1.4802 - val_accuracy: 0.3788
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4127 - accuracy: 0.3817 - val_loss: 1.4504 - val_accuracy: 0.3333
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3549 - accuracy: 0.4313 - val_loss: 1.4281 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3060 - accuracy: 0.5038 - val_loss: 1.4130 - val_accuracy: 0.3485
Epoch 7/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2594 - accuracy: 0.4809 - val_loss: 1.4022 - val_accuracy: 0.3636
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2283 - accuracy: 0.5344 - val_loss: 1.3922 - val_accuracy: 0.3788
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1966 - accuracy: 0.5344 - val_loss: 1.3790 - val_accuracy: 0.3636
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1407 - accuracy: 0.6069 - val_loss: 1.3654 - val_accuracy: 0.3939
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1179 - accuracy: 0.5802 - val_loss: 1.3575 - val_accuracy: 0.3788
Epoch 12/100
6/6 [==============================] - 0s 8ms/step - loss: 1.0753 - accuracy: 0.6298 - val_loss: 1.3402 - val_accuracy: 0.3788
Epoch 13/100
6/6 [==============================] - 0s 8ms/step - loss: 1.0283 - accuracy: 0.6183 - val_loss: 1.3329 - val_accuracy: 0.3939
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9835 - accuracy: 0.6336 - val_loss: 1.3306 - val_accuracy: 0.3939
Epoch 15/100
6/6 [==============================] - 0s 8ms/step - loss: 0.9426 - accuracy: 0.6794 - val_loss: 1.3258 - val_accuracy: 0.3939
Epoch 16/100
6/6 [==============================] - 0s 9ms/step - loss: 0.8962 - accuracy: 0.6870 - val_loss: 1.3274 - val_accuracy: 0.4242
Epoch 17/100
6/6 [==============================] - 0s 11ms/step - loss: 0.8593 - accuracy: 0.7137 - val_loss: 1.3131 - val_accuracy: 0.4242
Epoch 18/100
6/6 [==============================] - 0s 8ms/step - loss: 0.8000 - accuracy: 0.7366 - val_loss: 1.3188 - val_accuracy: 0.4394
Epoch 19/100
6/6 [==============================] - 0s 9ms/step - loss: 0.7522 - accuracy: 0.7786 - val_loss: 1.3268 - val_accuracy: 0.4545
Epoch 20/100
6/6 [==============================] - 0s 7ms/step - loss: 0.6978 - accuracy: 0.7977 - val_loss: 1.3126 - val_accuracy: 0.4394
Epoch 21/100
6/6 [==============================] - 0s 8ms/step - loss: 0.6385 - accuracy: 0.8359 - val_loss: 1.3164 - val_accuracy: 0.4394
Epoch 22/100
6/6 [==============================] - 0s 11ms/step - loss: 0.5863 - accuracy: 0.8435 - val_loss: 1.3461 - val_accuracy: 0.3939
Epoch 23/100
6/6 [==============================] - 0s 8ms/step - loss: 0.5656 - accuracy: 0.8359 - val_loss: 1.3604 - val_accuracy: 0.4545
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_90"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_270 (Dense)           (None, 150)               33000     
                                                                 
 dropout_90 (Dropout)        (None, 150)               0         
                                                                 
 dense_271 (Dense)           (None, 50)                7550      
                                                                 
 dense_272 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 62ms/step - loss: 1.5690 - accuracy: 0.2901 - val_loss: 1.5578 - val_accuracy: 0.3030
Epoch 2/100
6/6 [==============================] - 0s 10ms/step - loss: 1.4875 - accuracy: 0.3626 - val_loss: 1.5194 - val_accuracy: 0.3182
Epoch 3/100
6/6 [==============================] - 0s 10ms/step - loss: 1.4369 - accuracy: 0.3435 - val_loss: 1.4961 - val_accuracy: 0.3333
Epoch 4/100
6/6 [==============================] - 0s 10ms/step - loss: 1.3904 - accuracy: 0.3626 - val_loss: 1.4822 - val_accuracy: 0.2727
Epoch 5/100
6/6 [==============================] - 0s 10ms/step - loss: 1.3527 - accuracy: 0.3855 - val_loss: 1.4708 - val_accuracy: 0.3182
Epoch 6/100
6/6 [==============================] - 0s 14ms/step - loss: 1.3136 - accuracy: 0.4198 - val_loss: 1.4637 - val_accuracy: 0.3485
Epoch 7/100
6/6 [==============================] - 0s 19ms/step - loss: 1.2691 - accuracy: 0.4885 - val_loss: 1.4537 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 33ms/step - loss: 1.2366 - accuracy: 0.5115 - val_loss: 1.4437 - val_accuracy: 0.3485
Epoch 9/100
6/6 [==============================] - 0s 28ms/step - loss: 1.1997 - accuracy: 0.5687 - val_loss: 1.4342 - val_accuracy: 0.3485
Epoch 10/100
6/6 [==============================] - 0s 27ms/step - loss: 1.1527 - accuracy: 0.6069 - val_loss: 1.4211 - val_accuracy: 0.3788
Epoch 11/100
6/6 [==============================] - 0s 30ms/step - loss: 1.1224 - accuracy: 0.6069 - val_loss: 1.4066 - val_accuracy: 0.3939
Epoch 12/100
6/6 [==============================] - 0s 21ms/step - loss: 1.0568 - accuracy: 0.6489 - val_loss: 1.3752 - val_accuracy: 0.3788
Epoch 13/100
6/6 [==============================] - 0s 31ms/step - loss: 1.0218 - accuracy: 0.6718 - val_loss: 1.3628 - val_accuracy: 0.3788
Epoch 14/100
6/6 [==============================] - 0s 22ms/step - loss: 0.9637 - accuracy: 0.6985 - val_loss: 1.3684 - val_accuracy: 0.3939
Epoch 15/100
6/6 [==============================] - 0s 21ms/step - loss: 0.9155 - accuracy: 0.7023 - val_loss: 1.3679 - val_accuracy: 0.3939
Epoch 16/100
6/6 [==============================] - 0s 21ms/step - loss: 0.8478 - accuracy: 0.7137 - val_loss: 1.3646 - val_accuracy: 0.3939
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 6ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_91"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_273 (Dense)           (None, 150)               33000     
                                                                 
 dropout_91 (Dropout)        (None, 150)               0         
                                                                 
 dense_274 (Dense)           (None, 50)                7550      
                                                                 
 dense_275 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 113ms/step - loss: 1.6054 - accuracy: 0.2405 - val_loss: 1.5407 - val_accuracy: 0.3788
Epoch 2/100
6/6 [==============================] - 0s 16ms/step - loss: 1.5039 - accuracy: 0.3359 - val_loss: 1.4934 - val_accuracy: 0.3636
Epoch 3/100
6/6 [==============================] - 0s 16ms/step - loss: 1.4314 - accuracy: 0.3702 - val_loss: 1.4722 - val_accuracy: 0.3636
Epoch 4/100
6/6 [==============================] - 0s 17ms/step - loss: 1.3923 - accuracy: 0.3817 - val_loss: 1.4555 - val_accuracy: 0.3485
Epoch 5/100
6/6 [==============================] - 0s 13ms/step - loss: 1.3436 - accuracy: 0.4122 - val_loss: 1.4410 - val_accuracy: 0.3485
Epoch 6/100
6/6 [==============================] - 0s 13ms/step - loss: 1.3039 - accuracy: 0.4733 - val_loss: 1.4290 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 20ms/step - loss: 1.2744 - accuracy: 0.5534 - val_loss: 1.4133 - val_accuracy: 0.4091
Epoch 8/100
6/6 [==============================] - 0s 12ms/step - loss: 1.2330 - accuracy: 0.5840 - val_loss: 1.3926 - val_accuracy: 0.4091
Epoch 9/100
6/6 [==============================] - 0s 12ms/step - loss: 1.1917 - accuracy: 0.6107 - val_loss: 1.3693 - val_accuracy: 0.3939
Epoch 10/100
6/6 [==============================] - 0s 13ms/step - loss: 1.1620 - accuracy: 0.6069 - val_loss: 1.3459 - val_accuracy: 0.4242
Epoch 11/100
6/6 [==============================] - 0s 13ms/step - loss: 1.1084 - accuracy: 0.6183 - val_loss: 1.3321 - val_accuracy: 0.4242
Epoch 12/100
6/6 [==============================] - 0s 18ms/step - loss: 1.0836 - accuracy: 0.6221 - val_loss: 1.3110 - val_accuracy: 0.4394
Epoch 13/100
6/6 [==============================] - 0s 23ms/step - loss: 1.0341 - accuracy: 0.6679 - val_loss: 1.2929 - val_accuracy: 0.4394
Epoch 14/100
6/6 [==============================] - 0s 17ms/step - loss: 0.9912 - accuracy: 0.6489 - val_loss: 1.2818 - val_accuracy: 0.4545
Epoch 15/100
6/6 [==============================] - 0s 13ms/step - loss: 0.9242 - accuracy: 0.7137 - val_loss: 1.2681 - val_accuracy: 0.4394
Epoch 16/100
6/6 [==============================] - 0s 17ms/step - loss: 0.8999 - accuracy: 0.7061 - val_loss: 1.2658 - val_accuracy: 0.4242
Epoch 17/100
6/6 [==============================] - 0s 26ms/step - loss: 0.8472 - accuracy: 0.7252 - val_loss: 1.2492 - val_accuracy: 0.4394
Epoch 18/100
6/6 [==============================] - 0s 12ms/step - loss: 0.7875 - accuracy: 0.7748 - val_loss: 1.2492 - val_accuracy: 0.4242
Epoch 19/100
6/6 [==============================] - 0s 11ms/step - loss: 0.7500 - accuracy: 0.7863 - val_loss: 1.2494 - val_accuracy: 0.4242
Epoch 20/100
6/6 [==============================] - 0s 12ms/step - loss: 0.6882 - accuracy: 0.7939 - val_loss: 1.2382 - val_accuracy: 0.3939
Epoch 21/100
6/6 [==============================] - 0s 18ms/step - loss: 0.6363 - accuracy: 0.8511 - val_loss: 1.2413 - val_accuracy: 0.4091
Epoch 22/100
6/6 [==============================] - 0s 12ms/step - loss: 0.5724 - accuracy: 0.8550 - val_loss: 1.2616 - val_accuracy: 0.4091
Epoch 23/100
6/6 [==============================] - 0s 13ms/step - loss: 0.5434 - accuracy: 0.8511 - val_loss: 1.2697 - val_accuracy: 0.4242
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 5ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 6ms/step
Model: "sequential_92"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_276 (Dense)           (None, 150)               33000     
                                                                 
 dropout_92 (Dropout)        (None, 150)               0         
                                                                 
 dense_277 (Dense)           (None, 50)                7550      
                                                                 
 dense_278 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 70ms/step - loss: 1.6366 - accuracy: 0.2214 - val_loss: 1.5814 - val_accuracy: 0.1970
Epoch 2/100
6/6 [==============================] - 0s 16ms/step - loss: 1.5484 - accuracy: 0.3053 - val_loss: 1.5338 - val_accuracy: 0.3182
Epoch 3/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4935 - accuracy: 0.4466 - val_loss: 1.4955 - val_accuracy: 0.3333
Epoch 4/100
6/6 [==============================] - 0s 13ms/step - loss: 1.4412 - accuracy: 0.5305 - val_loss: 1.4538 - val_accuracy: 0.3333
Epoch 5/100
6/6 [==============================] - 0s 15ms/step - loss: 1.3786 - accuracy: 0.5534 - val_loss: 1.4118 - val_accuracy: 0.3788
Epoch 6/100
6/6 [==============================] - 0s 12ms/step - loss: 1.3217 - accuracy: 0.5420 - val_loss: 1.3777 - val_accuracy: 0.4091
Epoch 7/100
6/6 [==============================] - 0s 13ms/step - loss: 1.2675 - accuracy: 0.5267 - val_loss: 1.3525 - val_accuracy: 0.3939
Epoch 8/100
6/6 [==============================] - 0s 16ms/step - loss: 1.2251 - accuracy: 0.5649 - val_loss: 1.3356 - val_accuracy: 0.4091
Epoch 9/100
6/6 [==============================] - 0s 12ms/step - loss: 1.1779 - accuracy: 0.5382 - val_loss: 1.3260 - val_accuracy: 0.4091
Epoch 10/100
6/6 [==============================] - 0s 12ms/step - loss: 1.1415 - accuracy: 0.5878 - val_loss: 1.3187 - val_accuracy: 0.3939
Epoch 11/100
6/6 [==============================] - 0s 12ms/step - loss: 1.1158 - accuracy: 0.5840 - val_loss: 1.3157 - val_accuracy: 0.4091
Epoch 12/100
6/6 [==============================] - 0s 14ms/step - loss: 1.0773 - accuracy: 0.6069 - val_loss: 1.2947 - val_accuracy: 0.4091
Epoch 13/100
6/6 [==============================] - 0s 13ms/step - loss: 1.0349 - accuracy: 0.6489 - val_loss: 1.2823 - val_accuracy: 0.3788
Epoch 14/100
6/6 [==============================] - 0s 12ms/step - loss: 0.9809 - accuracy: 0.6756 - val_loss: 1.2764 - val_accuracy: 0.4091
Epoch 15/100
6/6 [==============================] - 0s 12ms/step - loss: 0.9496 - accuracy: 0.6832 - val_loss: 1.2690 - val_accuracy: 0.4242
Epoch 16/100
6/6 [==============================] - 0s 17ms/step - loss: 0.9030 - accuracy: 0.7176 - val_loss: 1.2666 - val_accuracy: 0.4394
Epoch 17/100
6/6 [==============================] - 0s 17ms/step - loss: 0.8480 - accuracy: 0.7214 - val_loss: 1.2570 - val_accuracy: 0.4697
Epoch 18/100
6/6 [==============================] - 0s 21ms/step - loss: 0.8022 - accuracy: 0.7252 - val_loss: 1.2661 - val_accuracy: 0.4697
Epoch 19/100
6/6 [==============================] - 0s 18ms/step - loss: 0.7588 - accuracy: 0.7710 - val_loss: 1.2679 - val_accuracy: 0.4545
Epoch 20/100
6/6 [==============================] - 0s 22ms/step - loss: 0.7205 - accuracy: 0.8015 - val_loss: 1.2530 - val_accuracy: 0.4394
Epoch 21/100
6/6 [==============================] - 0s 21ms/step - loss: 0.6576 - accuracy: 0.8321 - val_loss: 1.2571 - val_accuracy: 0.4545
Epoch 22/100
6/6 [==============================] - 0s 24ms/step - loss: 0.6096 - accuracy: 0.8321 - val_loss: 1.2881 - val_accuracy: 0.4848
Epoch 23/100
6/6 [==============================] - 0s 20ms/step - loss: 0.5627 - accuracy: 0.8244 - val_loss: 1.3013 - val_accuracy: 0.4545
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 4ms/step
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 7ms/step
Model: "sequential_93"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_279 (Dense)           (None, 150)               33000     
                                                                 
 dropout_93 (Dropout)        (None, 150)               0         
                                                                 
 dense_280 (Dense)           (None, 50)                7550      
                                                                 
 dense_281 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 112ms/step - loss: 1.6375 - accuracy: 0.1489 - val_loss: 1.5623 - val_accuracy: 0.4091
Epoch 2/100
6/6 [==============================] - 0s 14ms/step - loss: 1.5435 - accuracy: 0.3626 - val_loss: 1.5063 - val_accuracy: 0.3485
Epoch 3/100
6/6 [==============================] - 0s 15ms/step - loss: 1.4773 - accuracy: 0.4237 - val_loss: 1.4640 - val_accuracy: 0.3788
Epoch 4/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4255 - accuracy: 0.4618 - val_loss: 1.4272 - val_accuracy: 0.3939
Epoch 5/100
6/6 [==============================] - 0s 15ms/step - loss: 1.3590 - accuracy: 0.5038 - val_loss: 1.3976 - val_accuracy: 0.3939
Epoch 6/100
6/6 [==============================] - 0s 14ms/step - loss: 1.2971 - accuracy: 0.5038 - val_loss: 1.3779 - val_accuracy: 0.3788
Epoch 7/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2628 - accuracy: 0.5038 - val_loss: 1.3648 - val_accuracy: 0.3788
Epoch 8/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2131 - accuracy: 0.5725 - val_loss: 1.3546 - val_accuracy: 0.3485
Epoch 9/100
6/6 [==============================] - 0s 9ms/step - loss: 1.1803 - accuracy: 0.5725 - val_loss: 1.3481 - val_accuracy: 0.3788
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1364 - accuracy: 0.6260 - val_loss: 1.3464 - val_accuracy: 0.4091
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1123 - accuracy: 0.5992 - val_loss: 1.3476 - val_accuracy: 0.4091
Epoch 12/100
6/6 [==============================] - 0s 8ms/step - loss: 1.0644 - accuracy: 0.6145 - val_loss: 1.3248 - val_accuracy: 0.3788
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0072 - accuracy: 0.6565 - val_loss: 1.3126 - val_accuracy: 0.3939
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 0.9813 - accuracy: 0.6756 - val_loss: 1.3108 - val_accuracy: 0.3939
Epoch 15/100
6/6 [==============================] - 0s 8ms/step - loss: 0.9382 - accuracy: 0.6794 - val_loss: 1.3084 - val_accuracy: 0.4091
Epoch 16/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8775 - accuracy: 0.7099 - val_loss: 1.3067 - val_accuracy: 0.4242
Epoch 17/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8305 - accuracy: 0.7595 - val_loss: 1.2987 - val_accuracy: 0.4848
Epoch 18/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7701 - accuracy: 0.7595 - val_loss: 1.3115 - val_accuracy: 0.4545
Epoch 19/100
6/6 [==============================] - 0s 8ms/step - loss: 0.7315 - accuracy: 0.7672 - val_loss: 1.3160 - val_accuracy: 0.4697
Epoch 20/100
6/6 [==============================] - 0s 9ms/step - loss: 0.6848 - accuracy: 0.8092 - val_loss: 1.3079 - val_accuracy: 0.4394
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 5ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_94"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_282 (Dense)           (None, 150)               33000     
                                                                 
 dropout_94 (Dropout)        (None, 150)               0         
                                                                 
 dense_283 (Dense)           (None, 50)                7550      
                                                                 
 dense_284 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 40ms/step - loss: 1.6185 - accuracy: 0.1412 - val_loss: 1.5655 - val_accuracy: 0.3030
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5378 - accuracy: 0.3855 - val_loss: 1.5185 - val_accuracy: 0.3485
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4739 - accuracy: 0.3893 - val_loss: 1.4837 - val_accuracy: 0.3182
Epoch 4/100
6/6 [==============================] - 0s 8ms/step - loss: 1.4271 - accuracy: 0.4237 - val_loss: 1.4531 - val_accuracy: 0.3485
Epoch 5/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3680 - accuracy: 0.4504 - val_loss: 1.4294 - val_accuracy: 0.3636
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3262 - accuracy: 0.4733 - val_loss: 1.4112 - val_accuracy: 0.3485
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2831 - accuracy: 0.5115 - val_loss: 1.3955 - val_accuracy: 0.3485
Epoch 8/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2411 - accuracy: 0.5115 - val_loss: 1.3807 - val_accuracy: 0.3939
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2102 - accuracy: 0.5191 - val_loss: 1.3628 - val_accuracy: 0.4242
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1667 - accuracy: 0.5725 - val_loss: 1.3452 - val_accuracy: 0.4091
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1431 - accuracy: 0.5763 - val_loss: 1.3347 - val_accuracy: 0.3788
Epoch 12/100
6/6 [==============================] - 0s 8ms/step - loss: 1.0880 - accuracy: 0.5916 - val_loss: 1.3244 - val_accuracy: 0.3939
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0455 - accuracy: 0.5992 - val_loss: 1.3149 - val_accuracy: 0.4091
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 1.0060 - accuracy: 0.6336 - val_loss: 1.3063 - val_accuracy: 0.3939
Epoch 15/100
6/6 [==============================] - 0s 8ms/step - loss: 0.9632 - accuracy: 0.6336 - val_loss: 1.2939 - val_accuracy: 0.4242
Epoch 16/100
6/6 [==============================] - 0s 8ms/step - loss: 0.9180 - accuracy: 0.6832 - val_loss: 1.2945 - val_accuracy: 0.4394
Epoch 17/100
6/6 [==============================] - 0s 8ms/step - loss: 0.8681 - accuracy: 0.6947 - val_loss: 1.2869 - val_accuracy: 0.4242
Epoch 18/100
6/6 [==============================] - 0s 7ms/step - loss: 0.8141 - accuracy: 0.7519 - val_loss: 1.2913 - val_accuracy: 0.4545
Epoch 19/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7597 - accuracy: 0.7710 - val_loss: 1.2927 - val_accuracy: 0.4545
Epoch 20/100
6/6 [==============================] - 0s 7ms/step - loss: 0.7059 - accuracy: 0.7939 - val_loss: 1.2796 - val_accuracy: 0.4242
Epoch 21/100
6/6 [==============================] - 0s 7ms/step - loss: 0.6484 - accuracy: 0.8092 - val_loss: 1.2868 - val_accuracy: 0.4545
Epoch 22/100
6/6 [==============================] - 0s 10ms/step - loss: 0.5954 - accuracy: 0.8511 - val_loss: 1.3130 - val_accuracy: 0.4697
Epoch 23/100
6/6 [==============================] - 0s 7ms/step - loss: 0.5569 - accuracy: 0.8588 - val_loss: 1.3296 - val_accuracy: 0.4091
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_95"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_285 (Dense)           (None, 150)               33000     
                                                                 
 dropout_95 (Dropout)        (None, 150)               0         
                                                                 
 dense_286 (Dense)           (None, 50)                7550      
                                                                 
 dense_287 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 63ms/step - loss: 1.6128 - accuracy: 0.2137 - val_loss: 1.5569 - val_accuracy: 0.3182
Epoch 2/100
6/6 [==============================] - 0s 12ms/step - loss: 1.5345 - accuracy: 0.3664 - val_loss: 1.5106 - val_accuracy: 0.3636
Epoch 3/100
6/6 [==============================] - 0s 15ms/step - loss: 1.4742 - accuracy: 0.4122 - val_loss: 1.4759 - val_accuracy: 0.3485
Epoch 4/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4131 - accuracy: 0.4656 - val_loss: 1.4428 - val_accuracy: 0.3939
Epoch 5/100
6/6 [==============================] - 0s 14ms/step - loss: 1.3489 - accuracy: 0.5153 - val_loss: 1.4119 - val_accuracy: 0.4091
Epoch 6/100
6/6 [==============================] - 0s 13ms/step - loss: 1.3017 - accuracy: 0.5191 - val_loss: 1.3897 - val_accuracy: 0.3939
Epoch 7/100
6/6 [==============================] - 0s 17ms/step - loss: 1.2605 - accuracy: 0.5191 - val_loss: 1.3760 - val_accuracy: 0.3939
Epoch 8/100
6/6 [==============================] - 0s 15ms/step - loss: 1.2165 - accuracy: 0.5191 - val_loss: 1.3686 - val_accuracy: 0.3939
Epoch 9/100
6/6 [==============================] - 0s 14ms/step - loss: 1.1885 - accuracy: 0.5420 - val_loss: 1.3610 - val_accuracy: 0.3939
Epoch 10/100
6/6 [==============================] - 0s 12ms/step - loss: 1.1540 - accuracy: 0.5916 - val_loss: 1.3500 - val_accuracy: 0.3939
Epoch 11/100
6/6 [==============================] - 0s 12ms/step - loss: 1.1219 - accuracy: 0.5687 - val_loss: 1.3415 - val_accuracy: 0.3636
Epoch 12/100
6/6 [==============================] - 0s 13ms/step - loss: 1.0740 - accuracy: 0.6298 - val_loss: 1.3227 - val_accuracy: 0.3788
Epoch 13/100
6/6 [==============================] - 0s 11ms/step - loss: 1.0358 - accuracy: 0.6450 - val_loss: 1.3129 - val_accuracy: 0.3788
Epoch 14/100
6/6 [==============================] - 0s 12ms/step - loss: 0.9946 - accuracy: 0.6489 - val_loss: 1.3132 - val_accuracy: 0.3788
Epoch 15/100
6/6 [==============================] - 0s 12ms/step - loss: 0.9305 - accuracy: 0.6756 - val_loss: 1.3098 - val_accuracy: 0.3788
Epoch 16/100
6/6 [==============================] - 0s 16ms/step - loss: 0.8967 - accuracy: 0.6794 - val_loss: 1.3139 - val_accuracy: 0.3636
Epoch 17/100
6/6 [==============================] - 0s 17ms/step - loss: 0.8746 - accuracy: 0.7252 - val_loss: 1.3087 - val_accuracy: 0.3636
Epoch 18/100
6/6 [==============================] - 0s 13ms/step - loss: 0.8159 - accuracy: 0.7366 - val_loss: 1.3207 - val_accuracy: 0.3636
Epoch 19/100
6/6 [==============================] - 0s 13ms/step - loss: 0.7708 - accuracy: 0.7710 - val_loss: 1.3288 - val_accuracy: 0.3636
Epoch 20/100
6/6 [==============================] - 0s 13ms/step - loss: 0.7073 - accuracy: 0.7939 - val_loss: 1.3179 - val_accuracy: 0.3939
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 6ms/step
11/11 [==============================] - 0s 4ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_96"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_288 (Dense)           (None, 150)               33000     
                                                                 
 dropout_96 (Dropout)        (None, 150)               0         
                                                                 
 dense_289 (Dense)           (None, 50)                7550      
                                                                 
 dense_290 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 101ms/step - loss: 1.5781 - accuracy: 0.2557 - val_loss: 1.5646 - val_accuracy: 0.1818
Epoch 2/100
6/6 [==============================] - 0s 24ms/step - loss: 1.4955 - accuracy: 0.3244 - val_loss: 1.5176 - val_accuracy: 0.2576
Epoch 3/100
6/6 [==============================] - 0s 26ms/step - loss: 1.4308 - accuracy: 0.4466 - val_loss: 1.4841 - val_accuracy: 0.3788
Epoch 4/100
6/6 [==============================] - 0s 22ms/step - loss: 1.3722 - accuracy: 0.4542 - val_loss: 1.4553 - val_accuracy: 0.3788
Epoch 5/100
6/6 [==============================] - 0s 19ms/step - loss: 1.3228 - accuracy: 0.5000 - val_loss: 1.4331 - val_accuracy: 0.3788
Epoch 6/100
6/6 [==============================] - 0s 21ms/step - loss: 1.2704 - accuracy: 0.5000 - val_loss: 1.4146 - val_accuracy: 0.3636
Epoch 7/100
6/6 [==============================] - 0s 20ms/step - loss: 1.2391 - accuracy: 0.5076 - val_loss: 1.3921 - val_accuracy: 0.3939
Epoch 8/100
6/6 [==============================] - 0s 19ms/step - loss: 1.2058 - accuracy: 0.5153 - val_loss: 1.3713 - val_accuracy: 0.4091
Epoch 9/100
6/6 [==============================] - 0s 21ms/step - loss: 1.1675 - accuracy: 0.5305 - val_loss: 1.3530 - val_accuracy: 0.4242
Epoch 10/100
6/6 [==============================] - 0s 18ms/step - loss: 1.1337 - accuracy: 0.5573 - val_loss: 1.3380 - val_accuracy: 0.3939
Epoch 11/100
6/6 [==============================] - 0s 20ms/step - loss: 1.0874 - accuracy: 0.5840 - val_loss: 1.3286 - val_accuracy: 0.3939
Epoch 12/100
6/6 [==============================] - 0s 23ms/step - loss: 1.0595 - accuracy: 0.6107 - val_loss: 1.3127 - val_accuracy: 0.4091
Epoch 13/100
6/6 [==============================] - 0s 22ms/step - loss: 1.0082 - accuracy: 0.6412 - val_loss: 1.3009 - val_accuracy: 0.3939
Epoch 14/100
6/6 [==============================] - 0s 24ms/step - loss: 0.9677 - accuracy: 0.6603 - val_loss: 1.2923 - val_accuracy: 0.4242
Epoch 15/100
6/6 [==============================] - 0s 19ms/step - loss: 0.9304 - accuracy: 0.6985 - val_loss: 1.2809 - val_accuracy: 0.4091
Epoch 16/100
6/6 [==============================] - 0s 20ms/step - loss: 0.9016 - accuracy: 0.6832 - val_loss: 1.2828 - val_accuracy: 0.4242
Epoch 17/100
6/6 [==============================] - 0s 17ms/step - loss: 0.8456 - accuracy: 0.6985 - val_loss: 1.2737 - val_accuracy: 0.4394
Epoch 18/100
6/6 [==============================] - 0s 23ms/step - loss: 0.7862 - accuracy: 0.7634 - val_loss: 1.2814 - val_accuracy: 0.4394
Epoch 19/100
6/6 [==============================] - 0s 18ms/step - loss: 0.7549 - accuracy: 0.7634 - val_loss: 1.2869 - val_accuracy: 0.4545
Epoch 20/100
6/6 [==============================] - 0s 20ms/step - loss: 0.6773 - accuracy: 0.8130 - val_loss: 1.2771 - val_accuracy: 0.4394
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_97"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_291 (Dense)           (None, 150)               33000     
                                                                 
 dropout_97 (Dropout)        (None, 150)               0         
                                                                 
 dense_292 (Dense)           (None, 50)                7550      
                                                                 
 dense_293 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 56ms/step - loss: 1.6037 - accuracy: 0.2176 - val_loss: 1.5560 - val_accuracy: 0.3182
Epoch 2/100
6/6 [==============================] - 0s 43ms/step - loss: 1.5329 - accuracy: 0.3664 - val_loss: 1.5168 - val_accuracy: 0.3333
Epoch 3/100
6/6 [==============================] - 0s 22ms/step - loss: 1.4750 - accuracy: 0.3817 - val_loss: 1.4834 - val_accuracy: 0.3485
Epoch 4/100
6/6 [==============================] - 0s 13ms/step - loss: 1.4260 - accuracy: 0.4160 - val_loss: 1.4543 - val_accuracy: 0.3636
Epoch 5/100
6/6 [==============================] - 0s 27ms/step - loss: 1.3720 - accuracy: 0.4427 - val_loss: 1.4312 - val_accuracy: 0.3636
Epoch 6/100
6/6 [==============================] - 0s 14ms/step - loss: 1.3249 - accuracy: 0.4466 - val_loss: 1.4146 - val_accuracy: 0.3788
Epoch 7/100
6/6 [==============================] - 0s 16ms/step - loss: 1.2869 - accuracy: 0.4847 - val_loss: 1.3958 - val_accuracy: 0.4091
Epoch 8/100
6/6 [==============================] - 0s 19ms/step - loss: 1.2383 - accuracy: 0.5229 - val_loss: 1.3818 - val_accuracy: 0.4242
Epoch 9/100
6/6 [==============================] - 0s 17ms/step - loss: 1.2099 - accuracy: 0.5267 - val_loss: 1.3686 - val_accuracy: 0.4242
Epoch 10/100
6/6 [==============================] - 0s 20ms/step - loss: 1.1650 - accuracy: 0.5229 - val_loss: 1.3561 - val_accuracy: 0.4091
Epoch 11/100
6/6 [==============================] - 0s 24ms/step - loss: 1.1350 - accuracy: 0.5458 - val_loss: 1.3508 - val_accuracy: 0.4242
Epoch 12/100
6/6 [==============================] - 0s 15ms/step - loss: 1.0835 - accuracy: 0.5992 - val_loss: 1.3347 - val_accuracy: 0.4091
Epoch 13/100
6/6 [==============================] - 0s 25ms/step - loss: 1.0319 - accuracy: 0.6450 - val_loss: 1.3245 - val_accuracy: 0.4091
Epoch 14/100
6/6 [==============================] - 0s 26ms/step - loss: 1.0119 - accuracy: 0.6221 - val_loss: 1.3221 - val_accuracy: 0.4091
Epoch 15/100
6/6 [==============================] - 0s 24ms/step - loss: 0.9534 - accuracy: 0.6870 - val_loss: 1.3130 - val_accuracy: 0.4242
Epoch 16/100
6/6 [==============================] - 0s 22ms/step - loss: 0.8991 - accuracy: 0.7214 - val_loss: 1.3142 - val_accuracy: 0.3788
Epoch 17/100
6/6 [==============================] - 0s 13ms/step - loss: 0.8787 - accuracy: 0.7099 - val_loss: 1.3032 - val_accuracy: 0.4091
Epoch 18/100
6/6 [==============================] - 0s 16ms/step - loss: 0.8096 - accuracy: 0.7595 - val_loss: 1.3103 - val_accuracy: 0.3939
Epoch 19/100
6/6 [==============================] - 0s 14ms/step - loss: 0.7763 - accuracy: 0.7824 - val_loss: 1.3189 - val_accuracy: 0.4091
Epoch 20/100
6/6 [==============================] - 0s 13ms/step - loss: 0.7485 - accuracy: 0.7557 - val_loss: 1.3075 - val_accuracy: 0.3485
11/11 [==============================] - 0s 4ms/step
3/3 [==============================] - 0s 7ms/step
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 8ms/step
Model: "sequential_98"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_294 (Dense)           (None, 150)               33000     
                                                                 
 dropout_98 (Dropout)        (None, 150)               0         
                                                                 
 dense_295 (Dense)           (None, 50)                7550      
                                                                 
 dense_296 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 66ms/step - loss: 1.5885 - accuracy: 0.2405 - val_loss: 1.5533 - val_accuracy: 0.2273
Epoch 2/100
6/6 [==============================] - 0s 17ms/step - loss: 1.5053 - accuracy: 0.3855 - val_loss: 1.5030 - val_accuracy: 0.3485
Epoch 3/100
6/6 [==============================] - 0s 16ms/step - loss: 1.4395 - accuracy: 0.4656 - val_loss: 1.4665 - val_accuracy: 0.3485
Epoch 4/100
6/6 [==============================] - 0s 27ms/step - loss: 1.3908 - accuracy: 0.4580 - val_loss: 1.4358 - val_accuracy: 0.3485
Epoch 5/100
6/6 [==============================] - 0s 16ms/step - loss: 1.3180 - accuracy: 0.4924 - val_loss: 1.4121 - val_accuracy: 0.3788
Epoch 6/100
6/6 [==============================] - 0s 11ms/step - loss: 1.2702 - accuracy: 0.5000 - val_loss: 1.3948 - val_accuracy: 0.3939
Epoch 7/100
6/6 [==============================] - 0s 11ms/step - loss: 1.2370 - accuracy: 0.5305 - val_loss: 1.3771 - val_accuracy: 0.3788
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1866 - accuracy: 0.5458 - val_loss: 1.3575 - val_accuracy: 0.4091
Epoch 9/100
6/6 [==============================] - 0s 9ms/step - loss: 1.1494 - accuracy: 0.5496 - val_loss: 1.3340 - val_accuracy: 0.3939
Epoch 10/100
6/6 [==============================] - 0s 11ms/step - loss: 1.1282 - accuracy: 0.5725 - val_loss: 1.3174 - val_accuracy: 0.4091
Epoch 11/100
6/6 [==============================] - 0s 13ms/step - loss: 1.0883 - accuracy: 0.5534 - val_loss: 1.3074 - val_accuracy: 0.4242
Epoch 12/100
6/6 [==============================] - 0s 12ms/step - loss: 1.0416 - accuracy: 0.5878 - val_loss: 1.2882 - val_accuracy: 0.4091
Epoch 13/100
6/6 [==============================] - 0s 14ms/step - loss: 0.9904 - accuracy: 0.5992 - val_loss: 1.2734 - val_accuracy: 0.4242
Epoch 14/100
6/6 [==============================] - 0s 11ms/step - loss: 0.9522 - accuracy: 0.6527 - val_loss: 1.2665 - val_accuracy: 0.4242
Epoch 15/100
6/6 [==============================] - 0s 12ms/step - loss: 0.9061 - accuracy: 0.6870 - val_loss: 1.2564 - val_accuracy: 0.4242
Epoch 16/100
6/6 [==============================] - 0s 13ms/step - loss: 0.8545 - accuracy: 0.7061 - val_loss: 1.2549 - val_accuracy: 0.4394
Epoch 17/100
6/6 [==============================] - 0s 13ms/step - loss: 0.8090 - accuracy: 0.7405 - val_loss: 1.2461 - val_accuracy: 0.4394
Epoch 18/100
6/6 [==============================] - 0s 18ms/step - loss: 0.7589 - accuracy: 0.7519 - val_loss: 1.2614 - val_accuracy: 0.4545
Epoch 19/100
6/6 [==============================] - 0s 17ms/step - loss: 0.6987 - accuracy: 0.8130 - val_loss: 1.2723 - val_accuracy: 0.4697
Epoch 20/100
6/6 [==============================] - 0s 16ms/step - loss: 0.6761 - accuracy: 0.7824 - val_loss: 1.2559 - val_accuracy: 0.4394
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 3ms/step

NN_Model(X_train_tfidffull_smote, X_test_tfidffull, y_train_tfidffull_smote, y_test_tfidffull)

Model: "sequential_99"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_297 (Dense)           (None, 150)               33000     
                                                                 
 dropout_99 (Dropout)        (None, 150)               0         
                                                                 
 dense_298 (Dense)           (None, 50)                7550      
                                                                 
 dense_299 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 46ms/step - loss: 1.5628 - accuracy: 0.3636 - val_loss: 1.8742 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 15ms/step - loss: 1.4588 - accuracy: 0.5068 - val_loss: 2.0855 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 24ms/step - loss: 1.3555 - accuracy: 0.5523 - val_loss: 2.2009 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 37ms/step - loss: 1.2640 - accuracy: 0.5614 - val_loss: 2.2020 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 5ms/step
3/3 [==============================] - 0s 5ms/step
18/18 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 3ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.469091	0.361446	0.367534	0.290639

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(10):
  result=NN_Model(X_train_tfidffull_smote, X_test_tfidffull, y_train_tfidffull_smote, y_test_tfidffull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('F1 score')
plt.ylabel('F1 score')
plt.xlabel('epoch')
plt.show()

Model: "sequential_100"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_300 (Dense)           (None, 150)               33000     
                                                                 
 dropout_100 (Dropout)       (None, 150)               0         
                                                                 
 dense_301 (Dense)           (None, 50)                7550      
                                                                 
 dense_302 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 29ms/step - loss: 1.5340 - accuracy: 0.3545 - val_loss: 1.8667 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.4064 - accuracy: 0.4727 - val_loss: 2.0801 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 6ms/step - loss: 1.2953 - accuracy: 0.5295 - val_loss: 2.1834 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.2016 - accuracy: 0.5523 - val_loss: 2.1999 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_101"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_303 (Dense)           (None, 150)               33000     
                                                                 
 dropout_101 (Dropout)       (None, 150)               0         
                                                                 
 dense_304 (Dense)           (None, 50)                7550      
                                                                 
 dense_305 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 27ms/step - loss: 1.5711 - accuracy: 0.2932 - val_loss: 1.7976 - val_accuracy: 0.0091
Epoch 2/100
9/9 [==============================] - 0s 7ms/step - loss: 1.4618 - accuracy: 0.4614 - val_loss: 2.0047 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3549 - accuracy: 0.5091 - val_loss: 2.1734 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 8ms/step - loss: 1.2599 - accuracy: 0.5114 - val_loss: 2.2789 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_102"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_306 (Dense)           (None, 150)               33000     
                                                                 
 dropout_102 (Dropout)       (None, 150)               0         
                                                                 
 dense_307 (Dense)           (None, 50)                7550      
                                                                 
 dense_308 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 42ms/step - loss: 1.5494 - accuracy: 0.3932 - val_loss: 1.7877 - val_accuracy: 0.0364
Epoch 2/100
9/9 [==============================] - 0s 10ms/step - loss: 1.4189 - accuracy: 0.4568 - val_loss: 2.0094 - val_accuracy: 0.0273
Epoch 3/100
9/9 [==============================] - 0s 9ms/step - loss: 1.3063 - accuracy: 0.5136 - val_loss: 2.1916 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 8ms/step - loss: 1.2070 - accuracy: 0.5568 - val_loss: 2.2888 - val_accuracy: 0.0091
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 5ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_103"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_309 (Dense)           (None, 150)               33000     
                                                                 
 dropout_103 (Dropout)       (None, 150)               0         
                                                                 
 dense_310 (Dense)           (None, 50)                7550      
                                                                 
 dense_311 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 29ms/step - loss: 1.5523 - accuracy: 0.3205 - val_loss: 1.9140 - val_accuracy: 0.0182
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.4238 - accuracy: 0.4409 - val_loss: 2.1731 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 7ms/step - loss: 1.3249 - accuracy: 0.4864 - val_loss: 2.3076 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 6ms/step - loss: 1.2373 - accuracy: 0.5068 - val_loss: 2.3408 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_104"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_312 (Dense)           (None, 150)               33000     
                                                                 
 dropout_104 (Dropout)       (None, 150)               0         
                                                                 
 dense_313 (Dense)           (None, 50)                7550      
                                                                 
 dense_314 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 28ms/step - loss: 1.5823 - accuracy: 0.3114 - val_loss: 1.6917 - val_accuracy: 0.1364
Epoch 2/100
9/9 [==============================] - 0s 8ms/step - loss: 1.4849 - accuracy: 0.4682 - val_loss: 1.8481 - val_accuracy: 0.0727
Epoch 3/100
9/9 [==============================] - 0s 8ms/step - loss: 1.3774 - accuracy: 0.5295 - val_loss: 1.9731 - val_accuracy: 0.0818
Epoch 4/100
9/9 [==============================] - 0s 7ms/step - loss: 1.2701 - accuracy: 0.5705 - val_loss: 2.0818 - val_accuracy: 0.0364
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_105"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_315 (Dense)           (None, 150)               33000     
                                                                 
 dropout_105 (Dropout)       (None, 150)               0         
                                                                 
 dense_316 (Dense)           (None, 50)                7550      
                                                                 
 dense_317 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 29ms/step - loss: 1.5754 - accuracy: 0.2864 - val_loss: 1.7490 - val_accuracy: 0.1000
Epoch 2/100
9/9 [==============================] - 0s 6ms/step - loss: 1.4673 - accuracy: 0.4591 - val_loss: 1.9860 - val_accuracy: 0.0364
Epoch 3/100
9/9 [==============================] - 0s 6ms/step - loss: 1.3493 - accuracy: 0.5205 - val_loss: 2.2146 - val_accuracy: 0.0273
Epoch 4/100
9/9 [==============================] - 0s 6ms/step - loss: 1.2375 - accuracy: 0.5750 - val_loss: 2.3527 - val_accuracy: 0.0273
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 5ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_106"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_318 (Dense)           (None, 150)               33000     
                                                                 
 dropout_106 (Dropout)       (None, 150)               0         
                                                                 
 dense_319 (Dense)           (None, 50)                7550      
                                                                 
 dense_320 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 29ms/step - loss: 1.5371 - accuracy: 0.2977 - val_loss: 1.8963 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 8ms/step - loss: 1.4291 - accuracy: 0.4659 - val_loss: 2.1401 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 8ms/step - loss: 1.3289 - accuracy: 0.5023 - val_loss: 2.2978 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 6ms/step - loss: 1.2297 - accuracy: 0.5477 - val_loss: 2.3553 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 4ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_107"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_321 (Dense)           (None, 150)               33000     
                                                                 
 dropout_107 (Dropout)       (None, 150)               0         
                                                                 
 dense_322 (Dense)           (None, 50)                7550      
                                                                 
 dense_323 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 26ms/step - loss: 1.5854 - accuracy: 0.2750 - val_loss: 1.7278 - val_accuracy: 0.1000
Epoch 2/100
9/9 [==============================] - 0s 8ms/step - loss: 1.4752 - accuracy: 0.4159 - val_loss: 1.8915 - val_accuracy: 0.0727
Epoch 3/100
9/9 [==============================] - 0s 6ms/step - loss: 1.3693 - accuracy: 0.5023 - val_loss: 2.0529 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 7ms/step - loss: 1.2623 - accuracy: 0.5295 - val_loss: 2.1777 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_108"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_324 (Dense)           (None, 150)               33000     
                                                                 
 dropout_108 (Dropout)       (None, 150)               0         
                                                                 
 dense_325 (Dense)           (None, 50)                7550      
                                                                 
 dense_326 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 26ms/step - loss: 1.5483 - accuracy: 0.3295 - val_loss: 1.9347 - val_accuracy: 0.1455
Epoch 2/100
9/9 [==============================] - 0s 7ms/step - loss: 1.4215 - accuracy: 0.4818 - val_loss: 2.1197 - val_accuracy: 0.0818
Epoch 3/100
9/9 [==============================] - 0s 6ms/step - loss: 1.3054 - accuracy: 0.5568 - val_loss: 2.2596 - val_accuracy: 0.0364
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.2099 - accuracy: 0.5591 - val_loss: 2.3146 - val_accuracy: 0.0091
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 4ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_109"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_327 (Dense)           (None, 150)               33000     
                                                                 
 dropout_109 (Dropout)       (None, 150)               0         
                                                                 
 dense_328 (Dense)           (None, 50)                7550      
                                                                 
 dense_329 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 27ms/step - loss: 1.5751 - accuracy: 0.2659 - val_loss: 1.7672 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 6ms/step - loss: 1.4820 - accuracy: 0.4045 - val_loss: 1.9280 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3726 - accuracy: 0.5114 - val_loss: 2.0765 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 7ms/step - loss: 1.2698 - accuracy: 0.5273 - val_loss: 2.1780 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 7ms/step
18/18 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 5ms/step

Applying ANN function on Word2Vec dataset-

NN_Model(X_train_wv, X_test_wv, y_train_wv, y_test_wv)

Model: "sequential_44"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_132 (Dense)           (None, 150)               30150     
                                                                 
 dropout_44 (Dropout)        (None, 150)               0         
                                                                 
 dense_133 (Dense)           (None, 50)                7550      
                                                                 
 dense_134 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 69ms/step - loss: 1.6041 - accuracy: 0.2405 - val_loss: 1.5946 - val_accuracy: 0.3333
Epoch 2/100
6/6 [==============================] - 0s 16ms/step - loss: 1.5833 - accuracy: 0.3359 - val_loss: 1.5785 - val_accuracy: 0.3333
Epoch 3/100
6/6 [==============================] - 0s 12ms/step - loss: 1.5623 - accuracy: 0.3359 - val_loss: 1.5616 - val_accuracy: 0.3333
Epoch 4/100
6/6 [==============================] - 0s 13ms/step - loss: 1.5388 - accuracy: 0.3359 - val_loss: 1.5467 - val_accuracy: 0.3333
Epoch 5/100
6/6 [==============================] - 0s 12ms/step - loss: 1.5138 - accuracy: 0.3359 - val_loss: 1.5344 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4904 - accuracy: 0.3359 - val_loss: 1.5287 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4730 - accuracy: 0.3359 - val_loss: 1.5303 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 13ms/step - loss: 1.4655 - accuracy: 0.3359 - val_loss: 1.5371 - val_accuracy: 0.3333
Epoch 9/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4579 - accuracy: 0.3359 - val_loss: 1.5431 - val_accuracy: 0.3333
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 3ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.335366	0.337349	0.168449	0.170194

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(10):
  result=NN_Model(X_train_wv, X_test_wv, y_train_wv, y_test_wv)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('F1 score')
plt.ylabel('F1 score')
plt.xlabel('epoch')
plt.show()

Model: "sequential_45"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_135 (Dense)           (None, 150)               30150     
                                                                 
 dropout_45 (Dropout)        (None, 150)               0         
                                                                 
 dense_136 (Dense)           (None, 50)                7550      
                                                                 
 dense_137 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 70ms/step - loss: 1.6041 - accuracy: 0.2366 - val_loss: 1.5979 - val_accuracy: 0.2121
Epoch 2/100
6/6 [==============================] - 0s 13ms/step - loss: 1.5850 - accuracy: 0.2672 - val_loss: 1.5856 - val_accuracy: 0.2121
Epoch 3/100
6/6 [==============================] - 0s 12ms/step - loss: 1.5654 - accuracy: 0.3015 - val_loss: 1.5717 - val_accuracy: 0.2121
Epoch 4/100
6/6 [==============================] - 0s 12ms/step - loss: 1.5428 - accuracy: 0.3244 - val_loss: 1.5565 - val_accuracy: 0.3636
Epoch 5/100
6/6 [==============================] - 0s 11ms/step - loss: 1.5151 - accuracy: 0.3359 - val_loss: 1.5424 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 11ms/step - loss: 1.4871 - accuracy: 0.3282 - val_loss: 1.5336 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 14ms/step - loss: 1.4667 - accuracy: 0.3359 - val_loss: 1.5328 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4567 - accuracy: 0.3359 - val_loss: 1.5398 - val_accuracy: 0.3333
Epoch 9/100
6/6 [==============================] - 0s 16ms/step - loss: 1.4503 - accuracy: 0.3359 - val_loss: 1.5502 - val_accuracy: 0.3333
Epoch 10/100
6/6 [==============================] - 0s 15ms/step - loss: 1.4499 - accuracy: 0.3359 - val_loss: 1.5566 - val_accuracy: 0.3333
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_46"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_138 (Dense)           (None, 150)               30150     
                                                                 
 dropout_46 (Dropout)        (None, 150)               0         
                                                                 
 dense_139 (Dense)           (None, 50)                7550      
                                                                 
 dense_140 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 39ms/step - loss: 1.6023 - accuracy: 0.2366 - val_loss: 1.5942 - val_accuracy: 0.3333
Epoch 2/100
6/6 [==============================] - 0s 9ms/step - loss: 1.5823 - accuracy: 0.3359 - val_loss: 1.5803 - val_accuracy: 0.3333
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5629 - accuracy: 0.3244 - val_loss: 1.5654 - val_accuracy: 0.3333
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5387 - accuracy: 0.3359 - val_loss: 1.5503 - val_accuracy: 0.3333
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5118 - accuracy: 0.3321 - val_loss: 1.5363 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4845 - accuracy: 0.3359 - val_loss: 1.5285 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4635 - accuracy: 0.3359 - val_loss: 1.5294 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4549 - accuracy: 0.3359 - val_loss: 1.5388 - val_accuracy: 0.3333
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4512 - accuracy: 0.3359 - val_loss: 1.5494 - val_accuracy: 0.3333
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 6ms/step
Model: "sequential_47"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_141 (Dense)           (None, 150)               30150     
                                                                 
 dropout_47 (Dropout)        (None, 150)               0         
                                                                 
 dense_142 (Dense)           (None, 50)                7550      
                                                                 
 dense_143 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 72ms/step - loss: 1.5994 - accuracy: 0.3092 - val_loss: 1.5918 - val_accuracy: 0.3333
Epoch 2/100
6/6 [==============================] - 0s 12ms/step - loss: 1.5764 - accuracy: 0.3359 - val_loss: 1.5767 - val_accuracy: 0.3333
Epoch 3/100
6/6 [==============================] - 0s 18ms/step - loss: 1.5539 - accuracy: 0.3359 - val_loss: 1.5616 - val_accuracy: 0.3333
Epoch 4/100
6/6 [==============================] - 0s 17ms/step - loss: 1.5309 - accuracy: 0.3359 - val_loss: 1.5477 - val_accuracy: 0.3333
Epoch 5/100
6/6 [==============================] - 0s 22ms/step - loss: 1.5028 - accuracy: 0.3359 - val_loss: 1.5362 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4766 - accuracy: 0.3359 - val_loss: 1.5307 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 16ms/step - loss: 1.4593 - accuracy: 0.3359 - val_loss: 1.5331 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 11ms/step - loss: 1.4519 - accuracy: 0.3359 - val_loss: 1.5424 - val_accuracy: 0.3333
Epoch 9/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4511 - accuracy: 0.3359 - val_loss: 1.5530 - val_accuracy: 0.3333
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 5ms/step
11/11 [==============================] - 0s 4ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_48"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_144 (Dense)           (None, 150)               30150     
                                                                 
 dropout_48 (Dropout)        (None, 150)               0         
                                                                 
 dense_145 (Dense)           (None, 50)                7550      
                                                                 
 dense_146 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 88ms/step - loss: 1.5998 - accuracy: 0.3015 - val_loss: 1.5923 - val_accuracy: 0.3333
Epoch 2/100
6/6 [==============================] - 0s 16ms/step - loss: 1.5777 - accuracy: 0.3359 - val_loss: 1.5776 - val_accuracy: 0.3333
Epoch 3/100
6/6 [==============================] - 0s 12ms/step - loss: 1.5574 - accuracy: 0.3321 - val_loss: 1.5624 - val_accuracy: 0.3333
Epoch 4/100
6/6 [==============================] - 0s 16ms/step - loss: 1.5329 - accuracy: 0.3359 - val_loss: 1.5476 - val_accuracy: 0.3333
Epoch 5/100
6/6 [==============================] - 0s 17ms/step - loss: 1.5050 - accuracy: 0.3359 - val_loss: 1.5346 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 13ms/step - loss: 1.4772 - accuracy: 0.3359 - val_loss: 1.5284 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 11ms/step - loss: 1.4579 - accuracy: 0.3359 - val_loss: 1.5315 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4545 - accuracy: 0.3359 - val_loss: 1.5423 - val_accuracy: 0.3333
Epoch 9/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4537 - accuracy: 0.3359 - val_loss: 1.5532 - val_accuracy: 0.3333
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 5ms/step
Model: "sequential_49"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_147 (Dense)           (None, 150)               30150     
                                                                 
 dropout_49 (Dropout)        (None, 150)               0         
                                                                 
 dense_148 (Dense)           (None, 50)                7550      
                                                                 
 dense_149 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 65ms/step - loss: 1.5946 - accuracy: 0.2863 - val_loss: 1.5856 - val_accuracy: 0.2121
Epoch 2/100
6/6 [==============================] - 0s 15ms/step - loss: 1.5640 - accuracy: 0.3282 - val_loss: 1.5669 - val_accuracy: 0.3333
Epoch 3/100
6/6 [==============================] - 0s 33ms/step - loss: 1.5367 - accuracy: 0.3206 - val_loss: 1.5494 - val_accuracy: 0.3333
Epoch 4/100
6/6 [==============================] - 0s 21ms/step - loss: 1.5071 - accuracy: 0.3053 - val_loss: 1.5361 - val_accuracy: 0.3333
Epoch 5/100
6/6 [==============================] - 0s 38ms/step - loss: 1.4838 - accuracy: 0.3321 - val_loss: 1.5305 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 44ms/step - loss: 1.4609 - accuracy: 0.3321 - val_loss: 1.5347 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 29ms/step - loss: 1.4513 - accuracy: 0.3359 - val_loss: 1.5434 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 14ms/step - loss: 1.4592 - accuracy: 0.3359 - val_loss: 1.5507 - val_accuracy: 0.3333
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 4ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_50"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_150 (Dense)           (None, 150)               30150     
                                                                 
 dropout_50 (Dropout)        (None, 150)               0         
                                                                 
 dense_151 (Dense)           (None, 50)                7550      
                                                                 
 dense_152 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 39ms/step - loss: 1.6029 - accuracy: 0.2977 - val_loss: 1.5944 - val_accuracy: 0.3333
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5832 - accuracy: 0.3359 - val_loss: 1.5804 - val_accuracy: 0.3333
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5644 - accuracy: 0.3359 - val_loss: 1.5655 - val_accuracy: 0.3333
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5428 - accuracy: 0.3359 - val_loss: 1.5502 - val_accuracy: 0.3333
Epoch 5/100
6/6 [==============================] - 0s 8ms/step - loss: 1.5169 - accuracy: 0.3359 - val_loss: 1.5362 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4943 - accuracy: 0.3359 - val_loss: 1.5282 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 9ms/step - loss: 1.4742 - accuracy: 0.3359 - val_loss: 1.5275 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4695 - accuracy: 0.3359 - val_loss: 1.5329 - val_accuracy: 0.3333
Epoch 9/100
6/6 [==============================] - 0s 11ms/step - loss: 1.4627 - accuracy: 0.3359 - val_loss: 1.5388 - val_accuracy: 0.3333
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4555 - accuracy: 0.3359 - val_loss: 1.5421 - val_accuracy: 0.3333
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_51"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_153 (Dense)           (None, 150)               30150     
                                                                 
 dropout_51 (Dropout)        (None, 150)               0         
                                                                 
 dense_154 (Dense)           (None, 50)                7550      
                                                                 
 dense_155 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 41ms/step - loss: 1.6014 - accuracy: 0.2634 - val_loss: 1.5939 - val_accuracy: 0.2121
Epoch 2/100
6/6 [==============================] - 0s 8ms/step - loss: 1.5786 - accuracy: 0.2710 - val_loss: 1.5795 - val_accuracy: 0.2121
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5584 - accuracy: 0.2634 - val_loss: 1.5641 - val_accuracy: 0.2121
Epoch 4/100
6/6 [==============================] - 0s 11ms/step - loss: 1.5326 - accuracy: 0.2634 - val_loss: 1.5498 - val_accuracy: 0.2121
Epoch 5/100
6/6 [==============================] - 0s 8ms/step - loss: 1.5038 - accuracy: 0.2710 - val_loss: 1.5383 - val_accuracy: 0.2121
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4772 - accuracy: 0.2672 - val_loss: 1.5334 - val_accuracy: 0.2121
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4611 - accuracy: 0.3244 - val_loss: 1.5360 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4534 - accuracy: 0.3550 - val_loss: 1.5437 - val_accuracy: 0.3333
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4524 - accuracy: 0.3359 - val_loss: 1.5505 - val_accuracy: 0.3333
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_52"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_156 (Dense)           (None, 150)               30150     
                                                                 
 dropout_52 (Dropout)        (None, 150)               0         
                                                                 
 dense_157 (Dense)           (None, 50)                7550      
                                                                 
 dense_158 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 41ms/step - loss: 1.6033 - accuracy: 0.2939 - val_loss: 1.5994 - val_accuracy: 0.2121
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5878 - accuracy: 0.2748 - val_loss: 1.5886 - val_accuracy: 0.2121
Epoch 3/100
6/6 [==============================] - 0s 9ms/step - loss: 1.5713 - accuracy: 0.2901 - val_loss: 1.5767 - val_accuracy: 0.2121
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5531 - accuracy: 0.2710 - val_loss: 1.5646 - val_accuracy: 0.2121
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5288 - accuracy: 0.3053 - val_loss: 1.5521 - val_accuracy: 0.2121
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5053 - accuracy: 0.3206 - val_loss: 1.5416 - val_accuracy: 0.2121
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4815 - accuracy: 0.3130 - val_loss: 1.5354 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4683 - accuracy: 0.3359 - val_loss: 1.5342 - val_accuracy: 0.3333
Epoch 9/100
6/6 [==============================] - 0s 8ms/step - loss: 1.4531 - accuracy: 0.3359 - val_loss: 1.5401 - val_accuracy: 0.3333
Epoch 10/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4498 - accuracy: 0.3359 - val_loss: 1.5494 - val_accuracy: 0.3333
Epoch 11/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4456 - accuracy: 0.3359 - val_loss: 1.5572 - val_accuracy: 0.3333
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_53"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_159 (Dense)           (None, 150)               30150     
                                                                 
 dropout_53 (Dropout)        (None, 150)               0         
                                                                 
 dense_160 (Dense)           (None, 50)                7550      
                                                                 
 dense_161 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 2s 137ms/step - loss: 1.6053 - accuracy: 0.2290 - val_loss: 1.6008 - val_accuracy: 0.2121
Epoch 2/100
6/6 [==============================] - 0s 16ms/step - loss: 1.5883 - accuracy: 0.2710 - val_loss: 1.5908 - val_accuracy: 0.2121
Epoch 3/100
6/6 [==============================] - 0s 15ms/step - loss: 1.5719 - accuracy: 0.2710 - val_loss: 1.5794 - val_accuracy: 0.2121
Epoch 4/100
6/6 [==============================] - 0s 26ms/step - loss: 1.5509 - accuracy: 0.2710 - val_loss: 1.5665 - val_accuracy: 0.2121
Epoch 5/100
6/6 [==============================] - 0s 24ms/step - loss: 1.5248 - accuracy: 0.2710 - val_loss: 1.5537 - val_accuracy: 0.2121
Epoch 6/100
6/6 [==============================] - 0s 26ms/step - loss: 1.4967 - accuracy: 0.2710 - val_loss: 1.5447 - val_accuracy: 0.2121
Epoch 7/100
6/6 [==============================] - 0s 17ms/step - loss: 1.4736 - accuracy: 0.2710 - val_loss: 1.5413 - val_accuracy: 0.2121
Epoch 8/100
6/6 [==============================] - 0s 15ms/step - loss: 1.4633 - accuracy: 0.2901 - val_loss: 1.5434 - val_accuracy: 0.3333
Epoch 9/100
6/6 [==============================] - 0s 18ms/step - loss: 1.4486 - accuracy: 0.3511 - val_loss: 1.5484 - val_accuracy: 0.3333
Epoch 10/100
6/6 [==============================] - 0s 13ms/step - loss: 1.4485 - accuracy: 0.3359 - val_loss: 1.5544 - val_accuracy: 0.3333
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 7ms/step
11/11 [==============================] - 0s 5ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_54"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_162 (Dense)           (None, 150)               30150     
                                                                 
 dropout_54 (Dropout)        (None, 150)               0         
                                                                 
 dense_163 (Dense)           (None, 50)                7550      
                                                                 
 dense_164 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 77ms/step - loss: 1.6047 - accuracy: 0.3168 - val_loss: 1.5984 - val_accuracy: 0.3333
Epoch 2/100
6/6 [==============================] - 0s 17ms/step - loss: 1.5900 - accuracy: 0.3359 - val_loss: 1.5879 - val_accuracy: 0.3333
Epoch 3/100
6/6 [==============================] - 0s 14ms/step - loss: 1.5766 - accuracy: 0.3359 - val_loss: 1.5761 - val_accuracy: 0.3333
Epoch 4/100
6/6 [==============================] - 0s 13ms/step - loss: 1.5587 - accuracy: 0.3359 - val_loss: 1.5632 - val_accuracy: 0.3333
Epoch 5/100
6/6 [==============================] - 0s 13ms/step - loss: 1.5391 - accuracy: 0.3359 - val_loss: 1.5485 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 25ms/step - loss: 1.5160 - accuracy: 0.3359 - val_loss: 1.5355 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 24ms/step - loss: 1.4889 - accuracy: 0.3359 - val_loss: 1.5270 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 18ms/step - loss: 1.4733 - accuracy: 0.3359 - val_loss: 1.5262 - val_accuracy: 0.3333
Epoch 9/100
6/6 [==============================] - 0s 15ms/step - loss: 1.4605 - accuracy: 0.3359 - val_loss: 1.5328 - val_accuracy: 0.3333
Epoch 10/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4533 - accuracy: 0.3359 - val_loss: 1.5402 - val_accuracy: 0.3333
Epoch 11/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4506 - accuracy: 0.3359 - val_loss: 1.5478 - val_accuracy: 0.3333
11/11 [==============================] - 0s 4ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step

NN_Model(X_train_wv_smote, X_test_wv, y_train_wv_smote, y_test_wv)

Model: "sequential_110"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_330 (Dense)           (None, 150)               30150     
                                                                 
 dropout_110 (Dropout)       (None, 150)               0         
                                                                 
 dense_331 (Dense)           (None, 50)                7550      
                                                                 
 dense_332 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 44ms/step - loss: 1.6035 - accuracy: 0.2545 - val_loss: 1.6768 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 9ms/step - loss: 1.5854 - accuracy: 0.2500 - val_loss: 1.7817 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 10ms/step - loss: 1.5634 - accuracy: 0.2500 - val_loss: 1.9253 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 12ms/step - loss: 1.5427 - accuracy: 0.2500 - val_loss: 2.1100 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.2	0.228916	0.066667	0.085282

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(10):
  result=NN_Model(X_train_wv_smote, X_test_wv, y_train_wv_smote, y_test_wv)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('F1 score')
plt.ylabel('F1 score')
plt.xlabel('epoch')
plt.show()

Model: "sequential_121"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_363 (Dense)           (None, 150)               30150     
                                                                 
 dropout_121 (Dropout)       (None, 150)               0         
                                                                 
 dense_364 (Dense)           (None, 50)                7550      
                                                                 
 dense_365 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 28ms/step - loss: 1.6007 - accuracy: 0.1932 - val_loss: 1.6946 - val_accuracy: 0.2091
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.5805 - accuracy: 0.2523 - val_loss: 1.8349 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 6ms/step - loss: 1.5544 - accuracy: 0.2500 - val_loss: 2.0235 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.5324 - accuracy: 0.2500 - val_loss: 2.2440 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_122"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_366 (Dense)           (None, 150)               30150     
                                                                 
 dropout_122 (Dropout)       (None, 150)               0         
                                                                 
 dense_367 (Dense)           (None, 50)                7550      
                                                                 
 dense_368 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 39ms/step - loss: 1.6004 - accuracy: 0.2273 - val_loss: 1.7114 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 8ms/step - loss: 1.5775 - accuracy: 0.2500 - val_loss: 1.8520 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 10ms/step - loss: 1.5538 - accuracy: 0.2500 - val_loss: 2.0324 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 8ms/step - loss: 1.5356 - accuracy: 0.2500 - val_loss: 2.2380 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 5ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 5ms/step
Model: "sequential_123"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_369 (Dense)           (None, 150)               30150     
                                                                 
 dropout_123 (Dropout)       (None, 150)               0         
                                                                 
 dense_370 (Dense)           (None, 50)                7550      
                                                                 
 dense_371 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 40ms/step - loss: 1.6025 - accuracy: 0.2364 - val_loss: 1.6805 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 8ms/step - loss: 1.5855 - accuracy: 0.2500 - val_loss: 1.7752 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 5ms/step - loss: 1.5667 - accuracy: 0.2500 - val_loss: 1.8987 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 8ms/step - loss: 1.5535 - accuracy: 0.2500 - val_loss: 2.0323 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_124"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_372 (Dense)           (None, 150)               30150     
                                                                 
 dropout_124 (Dropout)       (None, 150)               0         
                                                                 
 dense_373 (Dense)           (None, 50)                7550      
                                                                 
 dense_374 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 25ms/step - loss: 1.6046 - accuracy: 0.2318 - val_loss: 1.6602 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 8ms/step - loss: 1.5917 - accuracy: 0.2295 - val_loss: 1.7378 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 8ms/step - loss: 1.5728 - accuracy: 0.2500 - val_loss: 1.8441 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 6ms/step - loss: 1.5537 - accuracy: 0.2500 - val_loss: 1.9889 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_125"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_375 (Dense)           (None, 150)               30150     
                                                                 
 dropout_125 (Dropout)       (None, 150)               0         
                                                                 
 dense_376 (Dense)           (None, 50)                7550      
                                                                 
 dense_377 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 27ms/step - loss: 1.5994 - accuracy: 0.2091 - val_loss: 1.7063 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 8ms/step - loss: 1.5773 - accuracy: 0.2341 - val_loss: 1.8374 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 7ms/step - loss: 1.5538 - accuracy: 0.2500 - val_loss: 2.0116 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 7ms/step - loss: 1.5352 - accuracy: 0.2114 - val_loss: 2.2106 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_126"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_378 (Dense)           (None, 150)               30150     
                                                                 
 dropout_126 (Dropout)       (None, 150)               0         
                                                                 
 dense_379 (Dense)           (None, 50)                7550      
                                                                 
 dense_380 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 44ms/step - loss: 1.6019 - accuracy: 0.2295 - val_loss: 1.6782 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 10ms/step - loss: 1.5854 - accuracy: 0.2477 - val_loss: 1.7694 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 10ms/step - loss: 1.5660 - accuracy: 0.2523 - val_loss: 1.8977 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 10ms/step - loss: 1.5467 - accuracy: 0.2409 - val_loss: 2.0562 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_127"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_381 (Dense)           (None, 150)               30150     
                                                                 
 dropout_127 (Dropout)       (None, 150)               0         
                                                                 
 dense_382 (Dense)           (None, 50)                7550      
                                                                 
 dense_383 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 43ms/step - loss: 1.5978 - accuracy: 0.2227 - val_loss: 1.7273 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 8ms/step - loss: 1.5728 - accuracy: 0.2500 - val_loss: 1.9061 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 11ms/step - loss: 1.5463 - accuracy: 0.2500 - val_loss: 2.1282 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 9ms/step - loss: 1.5276 - accuracy: 0.2500 - val_loss: 2.3414 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 6ms/step
Model: "sequential_128"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_384 (Dense)           (None, 150)               30150     
                                                                 
 dropout_128 (Dropout)       (None, 150)               0         
                                                                 
 dense_385 (Dense)           (None, 50)                7550      
                                                                 
 dense_386 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 65ms/step - loss: 1.6017 - accuracy: 0.2500 - val_loss: 1.6801 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 17ms/step - loss: 1.5845 - accuracy: 0.2386 - val_loss: 1.7885 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 15ms/step - loss: 1.5608 - accuracy: 0.2432 - val_loss: 1.9467 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 16ms/step - loss: 1.5394 - accuracy: 0.2386 - val_loss: 2.1520 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 2ms/step
18/18 [==============================] - 0s 4ms/step
3/3 [==============================] - 0s 5ms/step
Model: "sequential_129"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_387 (Dense)           (None, 150)               30150     
                                                                 
 dropout_129 (Dropout)       (None, 150)               0         
                                                                 
 dense_388 (Dense)           (None, 50)                7550      
                                                                 
 dense_389 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 39ms/step - loss: 1.6028 - accuracy: 0.2250 - val_loss: 1.6770 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 9ms/step - loss: 1.5856 - accuracy: 0.2500 - val_loss: 1.7788 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 8ms/step - loss: 1.5630 - accuracy: 0.2500 - val_loss: 1.9218 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 10ms/step - loss: 1.5429 - accuracy: 0.2500 - val_loss: 2.1071 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 5ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_130"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_390 (Dense)           (None, 150)               30150     
                                                                 
 dropout_130 (Dropout)       (None, 150)               0         
                                                                 
 dense_391 (Dense)           (None, 50)                7550      
                                                                 
 dense_392 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 37,955
Trainable params: 37,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 24ms/step - loss: 1.6020 - accuracy: 0.2455 - val_loss: 1.6719 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 7ms/step - loss: 1.5866 - accuracy: 0.2500 - val_loss: 1.7606 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 6ms/step - loss: 1.5680 - accuracy: 0.2500 - val_loss: 1.8841 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 6ms/step - loss: 1.5472 - accuracy: 0.2523 - val_loss: 2.0408 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step

Applying ANN function on Word2Vec full dataset-

NN_Model(X_train_wvfull, X_test_wvfull, y_train_wvfull, y_test_wvfull)

Model: "sequential_55"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_165 (Dense)           (None, 150)               33000     
                                                                 
 dropout_55 (Dropout)        (None, 150)               0         
                                                                 
 dense_166 (Dense)           (None, 50)                7550      
                                                                 
 dense_167 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 67ms/step - loss: 1.5830 - accuracy: 0.3130 - val_loss: 1.5400 - val_accuracy: 0.3939
Epoch 2/100
6/6 [==============================] - 0s 13ms/step - loss: 1.5266 - accuracy: 0.3855 - val_loss: 1.4970 - val_accuracy: 0.4091
Epoch 3/100
6/6 [==============================] - 0s 15ms/step - loss: 1.4737 - accuracy: 0.4275 - val_loss: 1.4691 - val_accuracy: 0.3939
Epoch 4/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4313 - accuracy: 0.4008 - val_loss: 1.4409 - val_accuracy: 0.3939
Epoch 5/100
6/6 [==============================] - 0s 11ms/step - loss: 1.3923 - accuracy: 0.4313 - val_loss: 1.4142 - val_accuracy: 0.3788
Epoch 6/100
6/6 [==============================] - 0s 11ms/step - loss: 1.3523 - accuracy: 0.4466 - val_loss: 1.3954 - val_accuracy: 0.3788
Epoch 7/100
6/6 [==============================] - 0s 41ms/step - loss: 1.3224 - accuracy: 0.4504 - val_loss: 1.3814 - val_accuracy: 0.3636
Epoch 8/100
6/6 [==============================] - 0s 39ms/step - loss: 1.3003 - accuracy: 0.4580 - val_loss: 1.3734 - val_accuracy: 0.3939
Epoch 9/100
6/6 [==============================] - 0s 37ms/step - loss: 1.2799 - accuracy: 0.4542 - val_loss: 1.3657 - val_accuracy: 0.3939
Epoch 10/100
6/6 [==============================] - 0s 34ms/step - loss: 1.2689 - accuracy: 0.4466 - val_loss: 1.3561 - val_accuracy: 0.4091
Epoch 11/100
6/6 [==============================] - 0s 23ms/step - loss: 1.2735 - accuracy: 0.4466 - val_loss: 1.3483 - val_accuracy: 0.3788
Epoch 12/100
6/6 [==============================] - 0s 15ms/step - loss: 1.2477 - accuracy: 0.4580 - val_loss: 1.3378 - val_accuracy: 0.4091
Epoch 13/100
6/6 [==============================] - 0s 13ms/step - loss: 1.2265 - accuracy: 0.4924 - val_loss: 1.3269 - val_accuracy: 0.4091
Epoch 14/100
6/6 [==============================] - 0s 12ms/step - loss: 1.2273 - accuracy: 0.4771 - val_loss: 1.3194 - val_accuracy: 0.3939
Epoch 15/100
6/6 [==============================] - 0s 17ms/step - loss: 1.2088 - accuracy: 0.4847 - val_loss: 1.3099 - val_accuracy: 0.4091
Epoch 16/100
6/6 [==============================] - 0s 11ms/step - loss: 1.2197 - accuracy: 0.4542 - val_loss: 1.3121 - val_accuracy: 0.3939
Epoch 17/100
6/6 [==============================] - 0s 35ms/step - loss: 1.2115 - accuracy: 0.4847 - val_loss: 1.3060 - val_accuracy: 0.4091
Epoch 18/100
6/6 [==============================] - 0s 37ms/step - loss: 1.1965 - accuracy: 0.4885 - val_loss: 1.3056 - val_accuracy: 0.4091
Epoch 19/100
6/6 [==============================] - 0s 29ms/step - loss: 1.2019 - accuracy: 0.4542 - val_loss: 1.3066 - val_accuracy: 0.4091
Epoch 20/100
6/6 [==============================] - 0s 31ms/step - loss: 1.1982 - accuracy: 0.4847 - val_loss: 1.2899 - val_accuracy: 0.3939
Epoch 21/100
6/6 [==============================] - 0s 36ms/step - loss: 1.1766 - accuracy: 0.4847 - val_loss: 1.2864 - val_accuracy: 0.3939
Epoch 22/100
6/6 [==============================] - 0s 21ms/step - loss: 1.1905 - accuracy: 0.4733 - val_loss: 1.2912 - val_accuracy: 0.4091
Epoch 23/100
6/6 [==============================] - 0s 12ms/step - loss: 1.1905 - accuracy: 0.4847 - val_loss: 1.2959 - val_accuracy: 0.4242
Epoch 24/100
6/6 [==============================] - 0s 16ms/step - loss: 1.1673 - accuracy: 0.4847 - val_loss: 1.2883 - val_accuracy: 0.3788
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.469512	0.385542	0.442713	0.348271

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(10):
  result=NN_Model(X_train_wvfull, X_test_wvfull, y_train_wvfull, y_test_wvfull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('F1 score')
plt.ylabel('F1 score')
plt.xlabel('epoch')
plt.show()

Model: "sequential_111"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_333 (Dense)           (None, 150)               33000     
                                                                 
 dropout_111 (Dropout)       (None, 150)               0         
                                                                 
 dense_334 (Dense)           (None, 50)                7550      
                                                                 
 dense_335 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 56ms/step - loss: 1.5860 - accuracy: 0.2634 - val_loss: 1.5497 - val_accuracy: 0.3485
Epoch 2/100
6/6 [==============================] - 0s 14ms/step - loss: 1.5126 - accuracy: 0.3702 - val_loss: 1.5100 - val_accuracy: 0.3485
Epoch 3/100
6/6 [==============================] - 0s 14ms/step - loss: 1.4573 - accuracy: 0.3931 - val_loss: 1.4881 - val_accuracy: 0.3636
Epoch 4/100
6/6 [==============================] - 0s 15ms/step - loss: 1.4183 - accuracy: 0.3855 - val_loss: 1.4732 - val_accuracy: 0.3333
Epoch 5/100
6/6 [==============================] - 0s 16ms/step - loss: 1.3760 - accuracy: 0.3893 - val_loss: 1.4626 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 12ms/step - loss: 1.3538 - accuracy: 0.4008 - val_loss: 1.4511 - val_accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 11ms/step - loss: 1.3412 - accuracy: 0.4198 - val_loss: 1.4293 - val_accuracy: 0.3939
Epoch 8/100
6/6 [==============================] - 0s 13ms/step - loss: 1.3172 - accuracy: 0.4733 - val_loss: 1.4071 - val_accuracy: 0.3788
Epoch 9/100
6/6 [==============================] - 0s 10ms/step - loss: 1.2873 - accuracy: 0.4771 - val_loss: 1.3885 - val_accuracy: 0.4091
Epoch 10/100
6/6 [==============================] - 0s 13ms/step - loss: 1.2757 - accuracy: 0.4580 - val_loss: 1.3699 - val_accuracy: 0.4242
Epoch 11/100
6/6 [==============================] - 0s 15ms/step - loss: 1.2609 - accuracy: 0.4389 - val_loss: 1.3585 - val_accuracy: 0.4091
Epoch 12/100
6/6 [==============================] - 0s 11ms/step - loss: 1.2593 - accuracy: 0.4771 - val_loss: 1.3411 - val_accuracy: 0.4091
Epoch 13/100
6/6 [==============================] - 0s 12ms/step - loss: 1.2259 - accuracy: 0.4847 - val_loss: 1.3297 - val_accuracy: 0.4091
Epoch 14/100
6/6 [==============================] - 0s 14ms/step - loss: 1.2179 - accuracy: 0.4847 - val_loss: 1.3230 - val_accuracy: 0.4091
Epoch 15/100
6/6 [==============================] - 0s 12ms/step - loss: 1.2124 - accuracy: 0.4885 - val_loss: 1.3111 - val_accuracy: 0.4091
Epoch 16/100
6/6 [==============================] - 0s 12ms/step - loss: 1.2177 - accuracy: 0.4771 - val_loss: 1.3078 - val_accuracy: 0.4242
Epoch 17/100
6/6 [==============================] - 0s 11ms/step - loss: 1.2114 - accuracy: 0.5000 - val_loss: 1.2947 - val_accuracy: 0.4242
Epoch 18/100
6/6 [==============================] - 0s 11ms/step - loss: 1.1931 - accuracy: 0.4885 - val_loss: 1.2937 - val_accuracy: 0.4242
Epoch 19/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1959 - accuracy: 0.4695 - val_loss: 1.2910 - val_accuracy: 0.4091
Epoch 20/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1895 - accuracy: 0.4962 - val_loss: 1.2700 - val_accuracy: 0.4242
Epoch 21/100
6/6 [==============================] - 0s 11ms/step - loss: 1.1797 - accuracy: 0.5038 - val_loss: 1.2658 - val_accuracy: 0.4242
Epoch 22/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1661 - accuracy: 0.5038 - val_loss: 1.2810 - val_accuracy: 0.4091
Epoch 23/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1726 - accuracy: 0.4809 - val_loss: 1.2851 - val_accuracy: 0.4091
Epoch 24/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1607 - accuracy: 0.4962 - val_loss: 1.2789 - val_accuracy: 0.4394
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_112"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_336 (Dense)           (None, 150)               33000     
                                                                 
 dropout_112 (Dropout)       (None, 150)               0         
                                                                 
 dense_337 (Dense)           (None, 50)                7550      
                                                                 
 dense_338 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 42ms/step - loss: 1.6124 - accuracy: 0.1718 - val_loss: 1.5552 - val_accuracy: 0.3788
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5271 - accuracy: 0.3435 - val_loss: 1.5086 - val_accuracy: 0.3485
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4605 - accuracy: 0.3702 - val_loss: 1.4741 - val_accuracy: 0.3636
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4080 - accuracy: 0.3969 - val_loss: 1.4475 - val_accuracy: 0.4394
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3639 - accuracy: 0.4160 - val_loss: 1.4324 - val_accuracy: 0.3788
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3308 - accuracy: 0.4351 - val_loss: 1.4228 - val_accuracy: 0.3939
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3140 - accuracy: 0.4656 - val_loss: 1.4063 - val_accuracy: 0.3788
Epoch 8/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2956 - accuracy: 0.4733 - val_loss: 1.3927 - val_accuracy: 0.3788
Epoch 9/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2740 - accuracy: 0.4618 - val_loss: 1.3804 - val_accuracy: 0.3939
Epoch 10/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2603 - accuracy: 0.4733 - val_loss: 1.3691 - val_accuracy: 0.3939
Epoch 11/100
6/6 [==============================] - 0s 9ms/step - loss: 1.2662 - accuracy: 0.4313 - val_loss: 1.3640 - val_accuracy: 0.3939
Epoch 12/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2490 - accuracy: 0.4695 - val_loss: 1.3500 - val_accuracy: 0.3939
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2225 - accuracy: 0.4771 - val_loss: 1.3415 - val_accuracy: 0.3939
Epoch 14/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2326 - accuracy: 0.4656 - val_loss: 1.3373 - val_accuracy: 0.3939
Epoch 15/100
6/6 [==============================] - 0s 9ms/step - loss: 1.2095 - accuracy: 0.4962 - val_loss: 1.3274 - val_accuracy: 0.4091
Epoch 16/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2160 - accuracy: 0.5000 - val_loss: 1.3237 - val_accuracy: 0.4091
Epoch 17/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2081 - accuracy: 0.4847 - val_loss: 1.3127 - val_accuracy: 0.4091
Epoch 18/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1979 - accuracy: 0.4847 - val_loss: 1.3151 - val_accuracy: 0.4091
Epoch 19/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1954 - accuracy: 0.4656 - val_loss: 1.3150 - val_accuracy: 0.4091
Epoch 20/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1925 - accuracy: 0.4733 - val_loss: 1.2941 - val_accuracy: 0.4394
Epoch 21/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1774 - accuracy: 0.4962 - val_loss: 1.2879 - val_accuracy: 0.4394
Epoch 22/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1767 - accuracy: 0.4924 - val_loss: 1.2992 - val_accuracy: 0.4091
Epoch 23/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1720 - accuracy: 0.5000 - val_loss: 1.2995 - val_accuracy: 0.4091
Epoch 24/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1632 - accuracy: 0.5153 - val_loss: 1.2892 - val_accuracy: 0.4394
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_113"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_339 (Dense)           (None, 150)               33000     
                                                                 
 dropout_113 (Dropout)       (None, 150)               0         
                                                                 
 dense_340 (Dense)           (None, 50)                7550      
                                                                 
 dense_341 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 44ms/step - loss: 1.6095 - accuracy: 0.1794 - val_loss: 1.5698 - val_accuracy: 0.3788
Epoch 2/100
6/6 [==============================] - 0s 8ms/step - loss: 1.5310 - accuracy: 0.3969 - val_loss: 1.5203 - val_accuracy: 0.3788
Epoch 3/100
6/6 [==============================] - 0s 8ms/step - loss: 1.4801 - accuracy: 0.3931 - val_loss: 1.4834 - val_accuracy: 0.3485
Epoch 4/100
6/6 [==============================] - 0s 8ms/step - loss: 1.4288 - accuracy: 0.4122 - val_loss: 1.4511 - val_accuracy: 0.3636
Epoch 5/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3832 - accuracy: 0.3969 - val_loss: 1.4294 - val_accuracy: 0.3636
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3465 - accuracy: 0.4198 - val_loss: 1.4182 - val_accuracy: 0.3939
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3193 - accuracy: 0.4351 - val_loss: 1.4055 - val_accuracy: 0.4091
Epoch 8/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2996 - accuracy: 0.4733 - val_loss: 1.3933 - val_accuracy: 0.3939
Epoch 9/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2838 - accuracy: 0.4542 - val_loss: 1.3782 - val_accuracy: 0.4091
Epoch 10/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2751 - accuracy: 0.4733 - val_loss: 1.3624 - val_accuracy: 0.3939
Epoch 11/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2655 - accuracy: 0.4427 - val_loss: 1.3468 - val_accuracy: 0.3788
Epoch 12/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2405 - accuracy: 0.4924 - val_loss: 1.3314 - val_accuracy: 0.3788
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2298 - accuracy: 0.4809 - val_loss: 1.3208 - val_accuracy: 0.4091
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2310 - accuracy: 0.4656 - val_loss: 1.3153 - val_accuracy: 0.4091
Epoch 15/100
6/6 [==============================] - 0s 9ms/step - loss: 1.2238 - accuracy: 0.4809 - val_loss: 1.3080 - val_accuracy: 0.4242
Epoch 16/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2121 - accuracy: 0.4962 - val_loss: 1.3062 - val_accuracy: 0.4091
Epoch 17/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2106 - accuracy: 0.4771 - val_loss: 1.2965 - val_accuracy: 0.4242
Epoch 18/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1902 - accuracy: 0.4656 - val_loss: 1.2932 - val_accuracy: 0.4242
Epoch 19/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1920 - accuracy: 0.4771 - val_loss: 1.2919 - val_accuracy: 0.4242
Epoch 20/100
6/6 [==============================] - 0s 11ms/step - loss: 1.1975 - accuracy: 0.4847 - val_loss: 1.2759 - val_accuracy: 0.4242
Epoch 21/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1876 - accuracy: 0.4924 - val_loss: 1.2705 - val_accuracy: 0.4091
Epoch 22/100
6/6 [==============================] - 0s 9ms/step - loss: 1.1790 - accuracy: 0.4924 - val_loss: 1.2789 - val_accuracy: 0.4242
Epoch 23/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1845 - accuracy: 0.4733 - val_loss: 1.2822 - val_accuracy: 0.3939
Epoch 24/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1702 - accuracy: 0.4924 - val_loss: 1.2748 - val_accuracy: 0.3939
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_114"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_342 (Dense)           (None, 150)               33000     
                                                                 
 dropout_114 (Dropout)       (None, 150)               0         
                                                                 
 dense_343 (Dense)           (None, 50)                7550      
                                                                 
 dense_344 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 39ms/step - loss: 1.5953 - accuracy: 0.2519 - val_loss: 1.5520 - val_accuracy: 0.3485
Epoch 2/100
6/6 [==============================] - 0s 7ms/step - loss: 1.5289 - accuracy: 0.3473 - val_loss: 1.4993 - val_accuracy: 0.3636
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4690 - accuracy: 0.3664 - val_loss: 1.4668 - val_accuracy: 0.3788
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4200 - accuracy: 0.3893 - val_loss: 1.4388 - val_accuracy: 0.4091
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3684 - accuracy: 0.4275 - val_loss: 1.4183 - val_accuracy: 0.3485
Epoch 6/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3416 - accuracy: 0.4351 - val_loss: 1.4080 - val_accuracy: 0.3636
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3160 - accuracy: 0.4313 - val_loss: 1.3991 - val_accuracy: 0.3636
Epoch 8/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2947 - accuracy: 0.4580 - val_loss: 1.3905 - val_accuracy: 0.3788
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2629 - accuracy: 0.4695 - val_loss: 1.3810 - val_accuracy: 0.3788
Epoch 10/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2695 - accuracy: 0.4618 - val_loss: 1.3720 - val_accuracy: 0.3788
Epoch 11/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2564 - accuracy: 0.4580 - val_loss: 1.3615 - val_accuracy: 0.3788
Epoch 12/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2446 - accuracy: 0.4618 - val_loss: 1.3447 - val_accuracy: 0.3788
Epoch 13/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2336 - accuracy: 0.4695 - val_loss: 1.3329 - val_accuracy: 0.3939
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2209 - accuracy: 0.5000 - val_loss: 1.3292 - val_accuracy: 0.3939
Epoch 15/100
6/6 [==============================] - 0s 10ms/step - loss: 1.2234 - accuracy: 0.4733 - val_loss: 1.3238 - val_accuracy: 0.4091
Epoch 16/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2298 - accuracy: 0.4656 - val_loss: 1.3220 - val_accuracy: 0.3788
Epoch 17/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2189 - accuracy: 0.4695 - val_loss: 1.3112 - val_accuracy: 0.4091
Epoch 18/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2041 - accuracy: 0.4885 - val_loss: 1.3063 - val_accuracy: 0.4242
Epoch 19/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1948 - accuracy: 0.4885 - val_loss: 1.3045 - val_accuracy: 0.4242
Epoch 20/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1900 - accuracy: 0.4885 - val_loss: 1.2912 - val_accuracy: 0.3939
Epoch 21/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1877 - accuracy: 0.4733 - val_loss: 1.2877 - val_accuracy: 0.4091
Epoch 22/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1748 - accuracy: 0.5038 - val_loss: 1.2979 - val_accuracy: 0.3939
Epoch 23/100
6/6 [==============================] - 0s 14ms/step - loss: 1.1879 - accuracy: 0.4656 - val_loss: 1.3001 - val_accuracy: 0.3788
Epoch 24/100
6/6 [==============================] - 0s 12ms/step - loss: 1.1626 - accuracy: 0.5076 - val_loss: 1.2963 - val_accuracy: 0.3788
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 5ms/step
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_115"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_345 (Dense)           (None, 150)               33000     
                                                                 
 dropout_115 (Dropout)       (None, 150)               0         
                                                                 
 dense_346 (Dense)           (None, 50)                7550      
                                                                 
 dense_347 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 43ms/step - loss: 1.5838 - accuracy: 0.3206 - val_loss: 1.5490 - val_accuracy: 0.3485
Epoch 2/100
6/6 [==============================] - 0s 8ms/step - loss: 1.5151 - accuracy: 0.3779 - val_loss: 1.5062 - val_accuracy: 0.3485
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4605 - accuracy: 0.3931 - val_loss: 1.4745 - val_accuracy: 0.3333
Epoch 4/100
6/6 [==============================] - 0s 10ms/step - loss: 1.4109 - accuracy: 0.4389 - val_loss: 1.4436 - val_accuracy: 0.3636
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3649 - accuracy: 0.4351 - val_loss: 1.4240 - val_accuracy: 0.3485
Epoch 6/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3334 - accuracy: 0.4160 - val_loss: 1.4131 - val_accuracy: 0.3788
Epoch 7/100
6/6 [==============================] - 0s 9ms/step - loss: 1.3141 - accuracy: 0.4466 - val_loss: 1.3981 - val_accuracy: 0.4091
Epoch 8/100
6/6 [==============================] - 0s 9ms/step - loss: 1.2899 - accuracy: 0.4427 - val_loss: 1.3863 - val_accuracy: 0.3939
Epoch 9/100
6/6 [==============================] - 0s 9ms/step - loss: 1.2763 - accuracy: 0.4542 - val_loss: 1.3762 - val_accuracy: 0.4091
Epoch 10/100
6/6 [==============================] - 0s 9ms/step - loss: 1.2792 - accuracy: 0.4580 - val_loss: 1.3641 - val_accuracy: 0.4394
Epoch 11/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2595 - accuracy: 0.4618 - val_loss: 1.3568 - val_accuracy: 0.3939
Epoch 12/100
6/6 [==============================] - 0s 11ms/step - loss: 1.2399 - accuracy: 0.4733 - val_loss: 1.3431 - val_accuracy: 0.4394
Epoch 13/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2334 - accuracy: 0.4542 - val_loss: 1.3323 - val_accuracy: 0.4242
Epoch 14/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2350 - accuracy: 0.4695 - val_loss: 1.3268 - val_accuracy: 0.4091
Epoch 15/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2246 - accuracy: 0.4847 - val_loss: 1.3166 - val_accuracy: 0.4242
Epoch 16/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2178 - accuracy: 0.4427 - val_loss: 1.3160 - val_accuracy: 0.4242
Epoch 17/100
6/6 [==============================] - 0s 10ms/step - loss: 1.2231 - accuracy: 0.4695 - val_loss: 1.3092 - val_accuracy: 0.4242
Epoch 18/100
6/6 [==============================] - 0s 9ms/step - loss: 1.1918 - accuracy: 0.4733 - val_loss: 1.3077 - val_accuracy: 0.4242
Epoch 19/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2001 - accuracy: 0.4695 - val_loss: 1.3064 - val_accuracy: 0.4242
Epoch 20/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1826 - accuracy: 0.4962 - val_loss: 1.2886 - val_accuracy: 0.4242
Epoch 21/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1777 - accuracy: 0.4885 - val_loss: 1.2820 - val_accuracy: 0.4394
Epoch 22/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1815 - accuracy: 0.5038 - val_loss: 1.2925 - val_accuracy: 0.4091
Epoch 23/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1826 - accuracy: 0.5038 - val_loss: 1.2980 - val_accuracy: 0.4091
Epoch 24/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1718 - accuracy: 0.5038 - val_loss: 1.2873 - val_accuracy: 0.4394
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_116"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_348 (Dense)           (None, 150)               33000     
                                                                 
 dropout_116 (Dropout)       (None, 150)               0         
                                                                 
 dense_349 (Dense)           (None, 50)                7550      
                                                                 
 dense_350 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 39ms/step - loss: 1.5756 - accuracy: 0.2137 - val_loss: 1.5256 - val_accuracy: 0.3030
Epoch 2/100
6/6 [==============================] - 0s 8ms/step - loss: 1.4772 - accuracy: 0.3550 - val_loss: 1.4799 - val_accuracy: 0.3182
Epoch 3/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4251 - accuracy: 0.3817 - val_loss: 1.4567 - val_accuracy: 0.3182
Epoch 4/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3914 - accuracy: 0.3550 - val_loss: 1.4454 - val_accuracy: 0.3182
Epoch 5/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3543 - accuracy: 0.4160 - val_loss: 1.4352 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3237 - accuracy: 0.4580 - val_loss: 1.4211 - val_accuracy: 0.3485
Epoch 7/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3087 - accuracy: 0.4618 - val_loss: 1.4020 - val_accuracy: 0.3939
Epoch 8/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2801 - accuracy: 0.4695 - val_loss: 1.3860 - val_accuracy: 0.3939
Epoch 9/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2780 - accuracy: 0.4466 - val_loss: 1.3708 - val_accuracy: 0.4091
Epoch 10/100
6/6 [==============================] - 0s 10ms/step - loss: 1.2598 - accuracy: 0.4618 - val_loss: 1.3612 - val_accuracy: 0.3788
Epoch 11/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2671 - accuracy: 0.4733 - val_loss: 1.3609 - val_accuracy: 0.3939
Epoch 12/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2459 - accuracy: 0.4580 - val_loss: 1.3454 - val_accuracy: 0.3939
Epoch 13/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2242 - accuracy: 0.4580 - val_loss: 1.3348 - val_accuracy: 0.4091
Epoch 14/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2303 - accuracy: 0.4618 - val_loss: 1.3340 - val_accuracy: 0.4091
Epoch 15/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2123 - accuracy: 0.4695 - val_loss: 1.3278 - val_accuracy: 0.4091
Epoch 16/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2070 - accuracy: 0.4924 - val_loss: 1.3285 - val_accuracy: 0.4091
Epoch 17/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1914 - accuracy: 0.5038 - val_loss: 1.3186 - val_accuracy: 0.4242
Epoch 18/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2042 - accuracy: 0.4771 - val_loss: 1.3200 - val_accuracy: 0.4242
Epoch 19/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1854 - accuracy: 0.4618 - val_loss: 1.3226 - val_accuracy: 0.4242
Epoch 20/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1722 - accuracy: 0.5191 - val_loss: 1.3056 - val_accuracy: 0.4091
Epoch 21/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1769 - accuracy: 0.5000 - val_loss: 1.3016 - val_accuracy: 0.4091
Epoch 22/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1725 - accuracy: 0.5000 - val_loss: 1.3136 - val_accuracy: 0.4091
Epoch 23/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1622 - accuracy: 0.4924 - val_loss: 1.3158 - val_accuracy: 0.4091
Epoch 24/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1593 - accuracy: 0.4962 - val_loss: 1.3079 - val_accuracy: 0.4242
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_117"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_351 (Dense)           (None, 150)               33000     
                                                                 
 dropout_117 (Dropout)       (None, 150)               0         
                                                                 
 dense_352 (Dense)           (None, 50)                7550      
                                                                 
 dense_353 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 39ms/step - loss: 1.5930 - accuracy: 0.2519 - val_loss: 1.5617 - val_accuracy: 0.3182
Epoch 2/100
6/6 [==============================] - 0s 8ms/step - loss: 1.5191 - accuracy: 0.3931 - val_loss: 1.5262 - val_accuracy: 0.3485
Epoch 3/100
6/6 [==============================] - 0s 8ms/step - loss: 1.4717 - accuracy: 0.4122 - val_loss: 1.5013 - val_accuracy: 0.3485
Epoch 4/100
6/6 [==============================] - 0s 7ms/step - loss: 1.4339 - accuracy: 0.4160 - val_loss: 1.4830 - val_accuracy: 0.3333
Epoch 5/100
6/6 [==============================] - 0s 8ms/step - loss: 1.4001 - accuracy: 0.4313 - val_loss: 1.4686 - val_accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3656 - accuracy: 0.4389 - val_loss: 1.4582 - val_accuracy: 0.3485
Epoch 7/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3450 - accuracy: 0.4466 - val_loss: 1.4416 - val_accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3280 - accuracy: 0.4656 - val_loss: 1.4273 - val_accuracy: 0.3333
Epoch 9/100
6/6 [==============================] - 0s 9ms/step - loss: 1.3075 - accuracy: 0.4466 - val_loss: 1.4155 - val_accuracy: 0.3636
Epoch 10/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2887 - accuracy: 0.4847 - val_loss: 1.3974 - val_accuracy: 0.3636
Epoch 11/100
6/6 [==============================] - 0s 13ms/step - loss: 1.2711 - accuracy: 0.4733 - val_loss: 1.3815 - val_accuracy: 0.3788
Epoch 12/100
6/6 [==============================] - 0s 9ms/step - loss: 1.2492 - accuracy: 0.4924 - val_loss: 1.3634 - val_accuracy: 0.3939
Epoch 13/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2506 - accuracy: 0.4656 - val_loss: 1.3468 - val_accuracy: 0.3939
Epoch 14/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2377 - accuracy: 0.4847 - val_loss: 1.3343 - val_accuracy: 0.3788
Epoch 15/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2153 - accuracy: 0.4809 - val_loss: 1.3215 - val_accuracy: 0.4091
Epoch 16/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2027 - accuracy: 0.5038 - val_loss: 1.3198 - val_accuracy: 0.4091
Epoch 17/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2052 - accuracy: 0.4924 - val_loss: 1.3097 - val_accuracy: 0.4091
Epoch 18/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2008 - accuracy: 0.5038 - val_loss: 1.3107 - val_accuracy: 0.4091
Epoch 19/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1918 - accuracy: 0.4847 - val_loss: 1.3122 - val_accuracy: 0.4091
Epoch 20/100
6/6 [==============================] - 0s 10ms/step - loss: 1.1856 - accuracy: 0.5000 - val_loss: 1.2923 - val_accuracy: 0.4091
Epoch 21/100
6/6 [==============================] - 0s 9ms/step - loss: 1.1831 - accuracy: 0.4847 - val_loss: 1.2865 - val_accuracy: 0.4091
Epoch 22/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1760 - accuracy: 0.4924 - val_loss: 1.2954 - val_accuracy: 0.4091
Epoch 23/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1786 - accuracy: 0.4847 - val_loss: 1.3022 - val_accuracy: 0.4242
Epoch 24/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1644 - accuracy: 0.5115 - val_loss: 1.2932 - val_accuracy: 0.4091
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_118"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_354 (Dense)           (None, 150)               33000     
                                                                 
 dropout_118 (Dropout)       (None, 150)               0         
                                                                 
 dense_355 (Dense)           (None, 50)                7550      
                                                                 
 dense_356 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 57ms/step - loss: 1.5923 - accuracy: 0.2061 - val_loss: 1.5526 - val_accuracy: 0.3485
Epoch 2/100
6/6 [==============================] - 0s 17ms/step - loss: 1.5226 - accuracy: 0.3473 - val_loss: 1.5048 - val_accuracy: 0.3939
Epoch 3/100
6/6 [==============================] - 0s 12ms/step - loss: 1.4698 - accuracy: 0.3817 - val_loss: 1.4713 - val_accuracy: 0.3939
Epoch 4/100
6/6 [==============================] - 0s 11ms/step - loss: 1.4193 - accuracy: 0.4122 - val_loss: 1.4438 - val_accuracy: 0.3939
Epoch 5/100
6/6 [==============================] - 0s 12ms/step - loss: 1.3750 - accuracy: 0.4160 - val_loss: 1.4223 - val_accuracy: 0.4091
Epoch 6/100
6/6 [==============================] - 0s 12ms/step - loss: 1.3422 - accuracy: 0.4389 - val_loss: 1.4059 - val_accuracy: 0.4091
Epoch 7/100
6/6 [==============================] - 0s 11ms/step - loss: 1.3164 - accuracy: 0.4389 - val_loss: 1.3922 - val_accuracy: 0.3939
Epoch 8/100
6/6 [==============================] - 0s 11ms/step - loss: 1.3041 - accuracy: 0.4351 - val_loss: 1.3805 - val_accuracy: 0.3939
Epoch 9/100
6/6 [==============================] - 0s 14ms/step - loss: 1.2871 - accuracy: 0.4313 - val_loss: 1.3679 - val_accuracy: 0.3788
Epoch 10/100
6/6 [==============================] - 0s 13ms/step - loss: 1.2730 - accuracy: 0.4695 - val_loss: 1.3557 - val_accuracy: 0.3788
Epoch 11/100
6/6 [==============================] - 0s 16ms/step - loss: 1.2684 - accuracy: 0.4237 - val_loss: 1.3477 - val_accuracy: 0.3939
Epoch 12/100
6/6 [==============================] - 0s 14ms/step - loss: 1.2497 - accuracy: 0.4580 - val_loss: 1.3362 - val_accuracy: 0.3939
Epoch 13/100
6/6 [==============================] - 0s 12ms/step - loss: 1.2502 - accuracy: 0.4656 - val_loss: 1.3258 - val_accuracy: 0.4091
Epoch 14/100
6/6 [==============================] - 0s 15ms/step - loss: 1.2363 - accuracy: 0.4542 - val_loss: 1.3213 - val_accuracy: 0.4091
Epoch 15/100
6/6 [==============================] - 0s 12ms/step - loss: 1.2252 - accuracy: 0.4427 - val_loss: 1.3148 - val_accuracy: 0.4242
Epoch 16/100
6/6 [==============================] - 0s 13ms/step - loss: 1.2190 - accuracy: 0.4618 - val_loss: 1.3178 - val_accuracy: 0.4242
Epoch 17/100
6/6 [==============================] - 0s 11ms/step - loss: 1.2195 - accuracy: 0.4580 - val_loss: 1.3116 - val_accuracy: 0.4091
Epoch 18/100
6/6 [==============================] - 0s 13ms/step - loss: 1.2052 - accuracy: 0.4618 - val_loss: 1.3085 - val_accuracy: 0.4242
Epoch 19/100
6/6 [==============================] - 0s 12ms/step - loss: 1.2161 - accuracy: 0.4733 - val_loss: 1.3048 - val_accuracy: 0.4242
Epoch 20/100
6/6 [==============================] - 0s 11ms/step - loss: 1.2040 - accuracy: 0.4924 - val_loss: 1.2903 - val_accuracy: 0.4091
Epoch 21/100
6/6 [==============================] - 0s 12ms/step - loss: 1.1833 - accuracy: 0.4733 - val_loss: 1.2875 - val_accuracy: 0.4091
Epoch 22/100
6/6 [==============================] - 0s 11ms/step - loss: 1.1803 - accuracy: 0.5000 - val_loss: 1.2948 - val_accuracy: 0.3939
Epoch 23/100
6/6 [==============================] - 0s 14ms/step - loss: 1.1819 - accuracy: 0.4885 - val_loss: 1.2915 - val_accuracy: 0.4091
Epoch 24/100
6/6 [==============================] - 0s 13ms/step - loss: 1.1734 - accuracy: 0.4809 - val_loss: 1.2847 - val_accuracy: 0.4242
Epoch 25/100
6/6 [==============================] - 0s 12ms/step - loss: 1.1591 - accuracy: 0.5000 - val_loss: 1.2926 - val_accuracy: 0.3788
Epoch 26/100
6/6 [==============================] - 0s 12ms/step - loss: 1.1715 - accuracy: 0.4809 - val_loss: 1.2946 - val_accuracy: 0.3939
Epoch 27/100
6/6 [==============================] - 0s 14ms/step - loss: 1.1518 - accuracy: 0.4962 - val_loss: 1.2914 - val_accuracy: 0.4091
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 5ms/step
Model: "sequential_119"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_357 (Dense)           (None, 150)               33000     
                                                                 
 dropout_119 (Dropout)       (None, 150)               0         
                                                                 
 dense_358 (Dense)           (None, 50)                7550      
                                                                 
 dense_359 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 40ms/step - loss: 1.5834 - accuracy: 0.2061 - val_loss: 1.5455 - val_accuracy: 0.1970
Epoch 2/100
6/6 [==============================] - 0s 8ms/step - loss: 1.5012 - accuracy: 0.2939 - val_loss: 1.4996 - val_accuracy: 0.3333
Epoch 3/100
6/6 [==============================] - 0s 8ms/step - loss: 1.4517 - accuracy: 0.3282 - val_loss: 1.4684 - val_accuracy: 0.3182
Epoch 4/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3998 - accuracy: 0.4122 - val_loss: 1.4470 - val_accuracy: 0.3636
Epoch 5/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3632 - accuracy: 0.4122 - val_loss: 1.4324 - val_accuracy: 0.3636
Epoch 6/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3386 - accuracy: 0.4084 - val_loss: 1.4231 - val_accuracy: 0.3636
Epoch 7/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3262 - accuracy: 0.4389 - val_loss: 1.4097 - val_accuracy: 0.4091
Epoch 8/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3039 - accuracy: 0.4427 - val_loss: 1.3960 - val_accuracy: 0.4091
Epoch 9/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2887 - accuracy: 0.4466 - val_loss: 1.3827 - val_accuracy: 0.4091
Epoch 10/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2686 - accuracy: 0.4695 - val_loss: 1.3675 - val_accuracy: 0.4091
Epoch 11/100
6/6 [==============================] - 0s 9ms/step - loss: 1.2726 - accuracy: 0.4771 - val_loss: 1.3570 - val_accuracy: 0.4091
Epoch 12/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2579 - accuracy: 0.4885 - val_loss: 1.3421 - val_accuracy: 0.4242
Epoch 13/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2376 - accuracy: 0.4847 - val_loss: 1.3325 - val_accuracy: 0.4091
Epoch 14/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2353 - accuracy: 0.4733 - val_loss: 1.3286 - val_accuracy: 0.4242
Epoch 15/100
6/6 [==============================] - 0s 11ms/step - loss: 1.2215 - accuracy: 0.4885 - val_loss: 1.3212 - val_accuracy: 0.4242
Epoch 16/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2231 - accuracy: 0.4809 - val_loss: 1.3183 - val_accuracy: 0.4242
Epoch 17/100
6/6 [==============================] - 0s 9ms/step - loss: 1.2105 - accuracy: 0.4962 - val_loss: 1.3022 - val_accuracy: 0.4242
Epoch 18/100
6/6 [==============================] - 0s 9ms/step - loss: 1.1972 - accuracy: 0.4847 - val_loss: 1.3003 - val_accuracy: 0.4091
Epoch 19/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1884 - accuracy: 0.4962 - val_loss: 1.3028 - val_accuracy: 0.4091
Epoch 20/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1885 - accuracy: 0.4809 - val_loss: 1.2859 - val_accuracy: 0.3939
Epoch 21/100
6/6 [==============================] - 0s 7ms/step - loss: 1.1831 - accuracy: 0.4809 - val_loss: 1.2789 - val_accuracy: 0.4091
Epoch 22/100
6/6 [==============================] - 0s 9ms/step - loss: 1.1804 - accuracy: 0.4771 - val_loss: 1.2924 - val_accuracy: 0.3939
Epoch 23/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1781 - accuracy: 0.4924 - val_loss: 1.3005 - val_accuracy: 0.3939
Epoch 24/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1708 - accuracy: 0.4885 - val_loss: 1.2931 - val_accuracy: 0.4091
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 5ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_120"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_360 (Dense)           (None, 150)               33000     
                                                                 
 dropout_120 (Dropout)       (None, 150)               0         
                                                                 
 dense_361 (Dense)           (None, 50)                7550      
                                                                 
 dense_362 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
6/6 [==============================] - 1s 40ms/step - loss: 1.5932 - accuracy: 0.2481 - val_loss: 1.5656 - val_accuracy: 0.2727
Epoch 2/100
6/6 [==============================] - 0s 8ms/step - loss: 1.5280 - accuracy: 0.2748 - val_loss: 1.5277 - val_accuracy: 0.2727
Epoch 3/100
6/6 [==============================] - 0s 8ms/step - loss: 1.4812 - accuracy: 0.3473 - val_loss: 1.4972 - val_accuracy: 0.2879
Epoch 4/100
6/6 [==============================] - 0s 11ms/step - loss: 1.4363 - accuracy: 0.4008 - val_loss: 1.4666 - val_accuracy: 0.2727
Epoch 5/100
6/6 [==============================] - 0s 7ms/step - loss: 1.3953 - accuracy: 0.3855 - val_loss: 1.4383 - val_accuracy: 0.3182
Epoch 6/100
6/6 [==============================] - 0s 10ms/step - loss: 1.3534 - accuracy: 0.4427 - val_loss: 1.4162 - val_accuracy: 0.3182
Epoch 7/100
6/6 [==============================] - 0s 8ms/step - loss: 1.3184 - accuracy: 0.4656 - val_loss: 1.3987 - val_accuracy: 0.3182
Epoch 8/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2996 - accuracy: 0.4275 - val_loss: 1.3877 - val_accuracy: 0.3485
Epoch 9/100
6/6 [==============================] - 0s 12ms/step - loss: 1.2913 - accuracy: 0.4466 - val_loss: 1.3768 - val_accuracy: 0.3636
Epoch 10/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2815 - accuracy: 0.4389 - val_loss: 1.3631 - val_accuracy: 0.3939
Epoch 11/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2733 - accuracy: 0.4542 - val_loss: 1.3532 - val_accuracy: 0.3788
Epoch 12/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2463 - accuracy: 0.4580 - val_loss: 1.3412 - val_accuracy: 0.3788
Epoch 13/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2390 - accuracy: 0.4542 - val_loss: 1.3327 - val_accuracy: 0.3788
Epoch 14/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2262 - accuracy: 0.4733 - val_loss: 1.3300 - val_accuracy: 0.3939
Epoch 15/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2204 - accuracy: 0.4809 - val_loss: 1.3208 - val_accuracy: 0.4091
Epoch 16/100
6/6 [==============================] - 0s 7ms/step - loss: 1.2096 - accuracy: 0.4733 - val_loss: 1.3161 - val_accuracy: 0.4242
Epoch 17/100
6/6 [==============================] - 0s 8ms/step - loss: 1.2126 - accuracy: 0.4618 - val_loss: 1.3055 - val_accuracy: 0.4242
Epoch 18/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1980 - accuracy: 0.4733 - val_loss: 1.3037 - val_accuracy: 0.4242
Epoch 19/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1990 - accuracy: 0.4809 - val_loss: 1.3037 - val_accuracy: 0.4242
Epoch 20/100
6/6 [==============================] - 0s 9ms/step - loss: 1.1905 - accuracy: 0.4771 - val_loss: 1.2905 - val_accuracy: 0.4242
Epoch 21/100
6/6 [==============================] - 0s 9ms/step - loss: 1.1815 - accuracy: 0.4847 - val_loss: 1.2877 - val_accuracy: 0.4242
Epoch 22/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1866 - accuracy: 0.4809 - val_loss: 1.2965 - val_accuracy: 0.4091
Epoch 23/100
6/6 [==============================] - 0s 9ms/step - loss: 1.1716 - accuracy: 0.4809 - val_loss: 1.2995 - val_accuracy: 0.4242
Epoch 24/100
6/6 [==============================] - 0s 8ms/step - loss: 1.1755 - accuracy: 0.4847 - val_loss: 1.2928 - val_accuracy: 0.4091
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step

NN_Model(X_train_wvfull_smote, X_test_wvfull, y_train_wvfull_smote, y_test_wvfull)

Model: "sequential_131"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_393 (Dense)           (None, 150)               33000     
                                                                 
 dropout_131 (Dropout)       (None, 150)               0         
                                                                 
 dense_394 (Dense)           (None, 50)                7550      
                                                                 
 dense_395 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 40ms/step - loss: 1.5469 - accuracy: 0.3045 - val_loss: 1.9001 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 13ms/step - loss: 1.4496 - accuracy: 0.4318 - val_loss: 2.1563 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 11ms/step - loss: 1.3630 - accuracy: 0.4364 - val_loss: 2.3253 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 12ms/step - loss: 1.2993 - accuracy: 0.4545 - val_loss: 2.4177 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 4ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.378182	0.373494	0.300109	0.297257

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(10):
  result=NN_Model(X_train_wvfull_smote, X_test_wvfull, y_train_wvfull_smote, y_test_wvfull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('F1 score')
plt.ylabel('F1 score')
plt.xlabel('epoch')
plt.show()

Model: "sequential_132"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_396 (Dense)           (None, 150)               33000     
                                                                 
 dropout_132 (Dropout)       (None, 150)               0         
                                                                 
 dense_397 (Dense)           (None, 50)                7550      
                                                                 
 dense_398 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 37ms/step - loss: 1.5866 - accuracy: 0.3000 - val_loss: 1.8459 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 9ms/step - loss: 1.4851 - accuracy: 0.4523 - val_loss: 2.1092 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 10ms/step - loss: 1.4096 - accuracy: 0.4455 - val_loss: 2.3097 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 11ms/step - loss: 1.3459 - accuracy: 0.4909 - val_loss: 2.4253 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step
Model: "sequential_133"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_399 (Dense)           (None, 150)               33000     
                                                                 
 dropout_133 (Dropout)       (None, 150)               0         
                                                                 
 dense_400 (Dense)           (None, 50)                7550      
                                                                 
 dense_401 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 37ms/step - loss: 1.5786 - accuracy: 0.3545 - val_loss: 1.7327 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 10ms/step - loss: 1.4952 - accuracy: 0.4114 - val_loss: 1.9344 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 8ms/step - loss: 1.4014 - accuracy: 0.4841 - val_loss: 2.0971 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3217 - accuracy: 0.4818 - val_loss: 2.2334 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 2ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_134"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_402 (Dense)           (None, 150)               33000     
                                                                 
 dropout_134 (Dropout)       (None, 150)               0         
                                                                 
 dense_403 (Dense)           (None, 50)                7550      
                                                                 
 dense_404 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 26ms/step - loss: 1.5805 - accuracy: 0.2591 - val_loss: 1.6967 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 6ms/step - loss: 1.4791 - accuracy: 0.4727 - val_loss: 1.8888 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3904 - accuracy: 0.4773 - val_loss: 2.0066 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3124 - accuracy: 0.4682 - val_loss: 2.1061 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_135"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_405 (Dense)           (None, 150)               33000     
                                                                 
 dropout_135 (Dropout)       (None, 150)               0         
                                                                 
 dense_406 (Dense)           (None, 50)                7550      
                                                                 
 dense_407 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 25ms/step - loss: 1.5652 - accuracy: 0.3250 - val_loss: 1.7454 - val_accuracy: 0.0364
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.4598 - accuracy: 0.4773 - val_loss: 1.9476 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 8ms/step - loss: 1.3691 - accuracy: 0.4614 - val_loss: 2.1410 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 8ms/step - loss: 1.2845 - accuracy: 0.4864 - val_loss: 2.3025 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_136"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_408 (Dense)           (None, 150)               33000     
                                                                 
 dropout_136 (Dropout)       (None, 150)               0         
                                                                 
 dense_409 (Dense)           (None, 50)                7550      
                                                                 
 dense_410 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 25ms/step - loss: 1.5773 - accuracy: 0.2705 - val_loss: 1.7628 - val_accuracy: 0.0636
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.4915 - accuracy: 0.4023 - val_loss: 1.9655 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 5ms/step - loss: 1.4070 - accuracy: 0.4455 - val_loss: 2.1819 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3334 - accuracy: 0.4614 - val_loss: 2.3715 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_137"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_411 (Dense)           (None, 150)               33000     
                                                                 
 dropout_137 (Dropout)       (None, 150)               0         
                                                                 
 dense_412 (Dense)           (None, 50)                7550      
                                                                 
 dense_413 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 28ms/step - loss: 1.5838 - accuracy: 0.2386 - val_loss: 1.7947 - val_accuracy: 0.0273
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.5000 - accuracy: 0.4273 - val_loss: 1.9920 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 6ms/step - loss: 1.4210 - accuracy: 0.4250 - val_loss: 2.2076 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3404 - accuracy: 0.4591 - val_loss: 2.3843 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_138"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_414 (Dense)           (None, 150)               33000     
                                                                 
 dropout_138 (Dropout)       (None, 150)               0         
                                                                 
 dense_415 (Dense)           (None, 50)                7550      
                                                                 
 dense_416 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 25ms/step - loss: 1.5414 - accuracy: 0.3750 - val_loss: 1.8231 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 5ms/step - loss: 1.4449 - accuracy: 0.4227 - val_loss: 2.0300 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 5ms/step - loss: 1.3596 - accuracy: 0.4432 - val_loss: 2.2172 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 5ms/step - loss: 1.2879 - accuracy: 0.4523 - val_loss: 2.3769 - val_accuracy: 0.0091
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_139"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_417 (Dense)           (None, 150)               33000     
                                                                 
 dropout_139 (Dropout)       (None, 150)               0         
                                                                 
 dense_418 (Dense)           (None, 50)                7550      
                                                                 
 dense_419 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 27ms/step - loss: 1.5689 - accuracy: 0.3545 - val_loss: 1.7840 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 6ms/step - loss: 1.4755 - accuracy: 0.4341 - val_loss: 1.9596 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 6ms/step - loss: 1.3851 - accuracy: 0.4500 - val_loss: 2.1082 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 6ms/step - loss: 1.3060 - accuracy: 0.4795 - val_loss: 2.2351 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 4ms/step
Model: "sequential_140"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_420 (Dense)           (None, 150)               33000     
                                                                 
 dropout_140 (Dropout)       (None, 150)               0         
                                                                 
 dense_421 (Dense)           (None, 50)                7550      
                                                                 
 dense_422 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 35ms/step - loss: 1.5602 - accuracy: 0.3273 - val_loss: 1.9251 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 10ms/step - loss: 1.4695 - accuracy: 0.4000 - val_loss: 2.2211 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 10ms/step - loss: 1.3961 - accuracy: 0.4364 - val_loss: 2.4134 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 8ms/step - loss: 1.3343 - accuracy: 0.4545 - val_loss: 2.4705 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 5ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
Model: "sequential_141"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_423 (Dense)           (None, 150)               33000     
                                                                 
 dropout_141 (Dropout)       (None, 150)               0         
                                                                 
 dense_424 (Dense)           (None, 50)                7550      
                                                                 
 dense_425 (Dense)           (None, 5)                 255       
                                                                 
=================================================================
Total params: 40,805
Trainable params: 40,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
9/9 [==============================] - 1s 36ms/step - loss: 1.5514 - accuracy: 0.3318 - val_loss: 1.9122 - val_accuracy: 0.0000e+00
Epoch 2/100
9/9 [==============================] - 0s 8ms/step - loss: 1.4767 - accuracy: 0.4409 - val_loss: 2.1277 - val_accuracy: 0.0000e+00
Epoch 3/100
9/9 [==============================] - 0s 6ms/step - loss: 1.4018 - accuracy: 0.4795 - val_loss: 2.2746 - val_accuracy: 0.0000e+00
Epoch 4/100
9/9 [==============================] - 0s 8ms/step - loss: 1.3431 - accuracy: 0.4705 - val_loss: 2.3576 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step

Observations-

a. The neural network is overfitted for count-vectorizer dataset.

Train accuracy- 74%, Test accuracy- 38%.

With full dataset-

Train accuracy- 84%, Test accuracy- 44%.

F1 test score is ranging between 30% to 42%.

b. The neural network is overfitted for TFIDF dataset.

Train accuracy- 53%, Test accuracy- 37%.

With full dataset-

Train accuracy- 77%, Test accuracy- 44%.

F1 test score is ranging between 30% to 42%.

c. The neural network is best with Word2Vec dataset. There is no overfitting. Accuracy is best for full dataset around 38%. F1 score is also comparatively good around 38%.

Train accuracy- 33%, Test accuracy- 33%.

With full dataset-

Train accuracy- 46%, Test accuracy- 38%.

F1 test score is ranging between 34% to 44%.

d. In general, all smote dataset are better fitted. Word2vec is giving best accuracy around 38% and F1 sccore around 30%.

d. There is need to apply gridsearch to find the best tuned parameters.

Parameters used-

Dropout as 20% Relu activation for hidden layers and softmax for output layer. Softmax is used in classification problems. Optimizer is Adam and loss function is categorical crossentropy

Applying tuning techniques on different dataset

CountVectorizer Dataset-

Tuned_ANN(X_cv_df, y_cv_df)

4/4 [==============================] - 2s 6ms/step - loss: 1.6024 - accuracy: 0.2249
1/1 [==============================] - 0s 270ms/step - loss: 1.5414 - accuracy: 0.3810
4/4 [==============================] - 1s 6ms/step - loss: 1.6238 - accuracy: 0.1946
1/1 [==============================] - 0s 234ms/step - loss: 1.6439 - accuracy: 0.2439
4/4 [==============================] - 1s 5ms/step - loss: 1.6638 - accuracy: 0.1784
1/1 [==============================] - 0s 250ms/step - loss: 1.5595 - accuracy: 0.3659
4/4 [==============================] - 1s 5ms/step - loss: 1.6946 - accuracy: 0.1459

WARNING:tensorflow:5 out of the last 16 calls to <function Model.make_test_function.<locals>.test_function at 0x7f77afb4e950> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

1/1 [==============================] - 0s 216ms/step - loss: 1.6288 - accuracy: 0.1463
4/4 [==============================] - 1s 6ms/step - loss: 1.6712 - accuracy: 0.1784

WARNING:tensorflow:6 out of the last 17 calls to <function Model.make_test_function.<locals>.test_function at 0x7f77bef235b0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

1/1 [==============================] - 0s 227ms/step - loss: 1.5420 - accuracy: 0.4146
4/4 [==============================] - 1s 6ms/step - loss: 1.5689 - accuracy: 0.2378
1/1 [==============================] - 0s 205ms/step - loss: 1.5213 - accuracy: 0.3171
4/4 [==============================] - 1s 7ms/step - loss: 1.5895 - accuracy: 0.2189
1/1 [==============================] - 0s 210ms/step - loss: 1.4902 - accuracy: 0.3171
4/4 [==============================] - 1s 3ms/step - loss: 1.6052 - accuracy: 0.2649
1/1 [==============================] - 0s 146ms/step - loss: 1.5733 - accuracy: 0.3415
4/4 [==============================] - 0s 3ms/step - loss: 1.6058 - accuracy: 0.2081
1/1 [==============================] - 0s 144ms/step - loss: 1.6607 - accuracy: 0.0976
4/4 [==============================] - 1s 9ms/step - loss: 1.6092 - accuracy: 0.2405
1/1 [==============================] - 0s 205ms/step - loss: 1.5513 - accuracy: 0.3171
4/4 [==============================] - 1s 5ms/step - loss: 1.5917 - accuracy: 0.2575
1/1 [==============================] - 0s 195ms/step - loss: 1.5262 - accuracy: 0.4286
4/4 [==============================] - 1s 5ms/step - loss: 1.6035 - accuracy: 0.2405
1/1 [==============================] - 0s 211ms/step - loss: 1.5239 - accuracy: 0.2927
4/4 [==============================] - 0s 4ms/step - loss: 1.5974 - accuracy: 0.2811
1/1 [==============================] - 0s 133ms/step - loss: 1.4694 - accuracy: 0.4390
4/4 [==============================] - 1s 4ms/step - loss: 1.5772 - accuracy: 0.2595
1/1 [==============================] - 0s 153ms/step - loss: 1.6363 - accuracy: 0.1951
4/4 [==============================] - 0s 4ms/step - loss: 1.6233 - accuracy: 0.2000
1/1 [==============================] - 0s 148ms/step - loss: 1.5468 - accuracy: 0.2683
4/4 [==============================] - 0s 4ms/step - loss: 1.5870 - accuracy: 0.2892
1/1 [==============================] - 0s 138ms/step - loss: 1.5248 - accuracy: 0.2195
4/4 [==============================] - 0s 5ms/step - loss: 1.6102 - accuracy: 0.2270
1/1 [==============================] - 0s 141ms/step - loss: 1.5098 - accuracy: 0.2439
4/4 [==============================] - 0s 5ms/step - loss: 1.5976 - accuracy: 0.2514
1/1 [==============================] - 0s 137ms/step - loss: 1.4946 - accuracy: 0.3902
4/4 [==============================] - 0s 4ms/step - loss: 1.7016 - accuracy: 0.1324
1/1 [==============================] - 0s 151ms/step - loss: 1.6866 - accuracy: 0.0732
4/4 [==============================] - 0s 4ms/step - loss: 1.5926 - accuracy: 0.2189
1/1 [==============================] - 0s 136ms/step - loss: 1.5432 - accuracy: 0.2927
4/4 [==============================] - 0s 5ms/step - loss: 1.5483 - accuracy: 0.2846
1/1 [==============================] - 0s 144ms/step - loss: 1.4731 - accuracy: 0.3571
4/4 [==============================] - 0s 4ms/step - loss: 1.5822 - accuracy: 0.2622
1/1 [==============================] - 0s 146ms/step - loss: 1.5338 - accuracy: 0.3415
4/4 [==============================] - 0s 4ms/step - loss: 1.5927 - accuracy: 0.2649
1/1 [==============================] - 0s 136ms/step - loss: 1.4323 - accuracy: 0.4146
4/4 [==============================] - 0s 5ms/step - loss: 1.5200 - accuracy: 0.3324
1/1 [==============================] - 0s 132ms/step - loss: 1.7390 - accuracy: 0.2195
4/4 [==============================] - 0s 5ms/step - loss: 1.5410 - accuracy: 0.3297
1/1 [==============================] - 0s 189ms/step - loss: 1.3809 - accuracy: 0.3415
4/4 [==============================] - 1s 8ms/step - loss: 1.5673 - accuracy: 0.2784
1/1 [==============================] - 0s 202ms/step - loss: 1.4064 - accuracy: 0.3902
4/4 [==============================] - 1s 7ms/step - loss: 1.5536 - accuracy: 0.2946
1/1 [==============================] - 0s 203ms/step - loss: 1.4354 - accuracy: 0.3415
4/4 [==============================] - 1s 5ms/step - loss: 1.5541 - accuracy: 0.3324
1/1 [==============================] - 0s 144ms/step - loss: 1.5783 - accuracy: 0.2927
4/4 [==============================] - 0s 4ms/step - loss: 1.5466 - accuracy: 0.3027
1/1 [==============================] - 0s 135ms/step - loss: 1.6873 - accuracy: 0.2927
4/4 [==============================] - 0s 4ms/step - loss: 1.5484 - accuracy: 0.3297
1/1 [==============================] - 0s 129ms/step - loss: 1.4361 - accuracy: 0.3659
4/4 [==============================] - 0s 5ms/step - loss: 1.5846 - accuracy: 0.2683
1/1 [==============================] - 0s 132ms/step - loss: 1.4140 - accuracy: 0.3571
4/4 [==============================] - 0s 5ms/step - loss: 1.5142 - accuracy: 0.3162
1/1 [==============================] - 0s 143ms/step - loss: 1.5905 - accuracy: 0.3415
4/4 [==============================] - 0s 5ms/step - loss: 1.5503 - accuracy: 0.2730
1/1 [==============================] - 0s 153ms/step - loss: 1.3403 - accuracy: 0.3902
4/4 [==============================] - 0s 5ms/step - loss: 1.5526 - accuracy: 0.2838
1/1 [==============================] - 0s 137ms/step - loss: 1.6402 - accuracy: 0.2195
4/4 [==============================] - 0s 5ms/step - loss: 1.5304 - accuracy: 0.3081
1/1 [==============================] - 0s 134ms/step - loss: 1.3414 - accuracy: 0.3171
4/4 [==============================] - 0s 5ms/step - loss: 1.5473 - accuracy: 0.2811
1/1 [==============================] - 0s 149ms/step - loss: 1.3392 - accuracy: 0.4390
4/4 [==============================] - 0s 5ms/step - loss: 1.5622 - accuracy: 0.2919
1/1 [==============================] - 0s 129ms/step - loss: 1.4417 - accuracy: 0.3171
4/4 [==============================] - 0s 5ms/step - loss: 1.5289 - accuracy: 0.2811
1/1 [==============================] - 0s 136ms/step - loss: 1.5689 - accuracy: 0.3171
4/4 [==============================] - 0s 5ms/step - loss: 1.5108 - accuracy: 0.2865
1/1 [==============================] - 0s 133ms/step - loss: 1.7264 - accuracy: 0.2927
4/4 [==============================] - 0s 5ms/step - loss: 1.5690 - accuracy: 0.2568
1/1 [==============================] - 0s 137ms/step - loss: 1.4287 - accuracy: 0.3659
4/4 [==============================] - 0s 4ms/step - loss: 1.6342 - accuracy: 0.1978
1/1 [==============================] - 0s 127ms/step - loss: 1.6321 - accuracy: 0.1667
4/4 [==============================] - 1s 4ms/step - loss: 1.6804 - accuracy: 0.1486
1/1 [==============================] - 0s 184ms/step - loss: 1.6622 - accuracy: 0.1463
4/4 [==============================] - 1s 4ms/step - loss: 1.6373 - accuracy: 0.2189
1/1 [==============================] - 0s 196ms/step - loss: 1.5938 - accuracy: 0.1463
4/4 [==============================] - 1s 6ms/step - loss: 1.6647 - accuracy: 0.2027
1/1 [==============================] - 0s 209ms/step - loss: 1.6356 - accuracy: 0.1951
4/4 [==============================] - 0s 3ms/step - loss: 1.6284 - accuracy: 0.2432
1/1 [==============================] - 0s 129ms/step - loss: 1.6039 - accuracy: 0.1463
4/4 [==============================] - 1s 4ms/step - loss: 1.6997 - accuracy: 0.1378
1/1 [==============================] - 0s 147ms/step - loss: 1.6523 - accuracy: 0.1220
4/4 [==============================] - 0s 4ms/step - loss: 1.5547 - accuracy: 0.2622
1/1 [==============================] - 0s 136ms/step - loss: 1.5114 - accuracy: 0.2439
4/4 [==============================] - 0s 4ms/step - loss: 1.5337 - accuracy: 0.3162
1/1 [==============================] - 0s 140ms/step - loss: 1.5652 - accuracy: 0.1951
4/4 [==============================] - 0s 4ms/step - loss: 1.5787 - accuracy: 0.2811
1/1 [==============================] - 0s 147ms/step - loss: 1.6870 - accuracy: 0.1951
4/4 [==============================] - 0s 4ms/step - loss: 1.5517 - accuracy: 0.2865
1/1 [==============================] - 0s 129ms/step - loss: 1.5058 - accuracy: 0.3902
4/4 [==============================] - 0s 4ms/step - loss: 1.6521 - accuracy: 0.2033
1/1 [==============================] - 0s 139ms/step - loss: 1.5535 - accuracy: 0.2619
4/4 [==============================] - 0s 4ms/step - loss: 1.5567 - accuracy: 0.2459
1/1 [==============================] - 0s 150ms/step - loss: 1.4068 - accuracy: 0.4634
4/4 [==============================] - 0s 4ms/step - loss: 1.5897 - accuracy: 0.2459
1/1 [==============================] - 0s 130ms/step - loss: 1.5396 - accuracy: 0.3415
4/4 [==============================] - 0s 5ms/step - loss: 1.5931 - accuracy: 0.2054
1/1 [==============================] - 0s 134ms/step - loss: 1.5720 - accuracy: 0.2927
4/4 [==============================] - 0s 4ms/step - loss: 1.5837 - accuracy: 0.2784
1/1 [==============================] - 0s 129ms/step - loss: 1.4920 - accuracy: 0.3415
4/4 [==============================] - 0s 4ms/step - loss: 1.5791 - accuracy: 0.2595
1/1 [==============================] - 0s 129ms/step - loss: 1.5291 - accuracy: 0.2927
4/4 [==============================] - 0s 5ms/step - loss: 1.6031 - accuracy: 0.1757
1/1 [==============================] - 0s 194ms/step - loss: 1.5399 - accuracy: 0.2683
4/4 [==============================] - 1s 5ms/step - loss: 1.6281 - accuracy: 0.2405
1/1 [==============================] - 0s 180ms/step - loss: 1.5603 - accuracy: 0.2439
4/4 [==============================] - 1s 5ms/step - loss: 1.6072 - accuracy: 0.2270
1/1 [==============================] - 0s 209ms/step - loss: 1.6608 - accuracy: 0.1463
4/4 [==============================] - 1s 4ms/step - loss: 1.6602 - accuracy: 0.1459
1/1 [==============================] - 0s 128ms/step - loss: 1.5545 - accuracy: 0.3659
4/4 [==============================] - 0s 4ms/step - loss: 1.5769 - accuracy: 0.2412
1/1 [==============================] - 0s 136ms/step - loss: 1.4706 - accuracy: 0.3095
4/4 [==============================] - 0s 4ms/step - loss: 1.5697 - accuracy: 0.2514
1/1 [==============================] - 0s 141ms/step - loss: 1.5458 - accuracy: 0.3415
4/4 [==============================] - 0s 4ms/step - loss: 1.5463 - accuracy: 0.3027
1/1 [==============================] - 0s 130ms/step - loss: 1.3950 - accuracy: 0.3902
4/4 [==============================] - 0s 4ms/step - loss: 1.5164 - accuracy: 0.3324
1/1 [==============================] - 0s 129ms/step - loss: 1.6723 - accuracy: 0.2195
4/4 [==============================] - 0s 5ms/step - loss: 1.6069 - accuracy: 0.2351
1/1 [==============================] - 0s 133ms/step - loss: 1.4554 - accuracy: 0.3171
4/4 [==============================] - 0s 5ms/step - loss: 1.5519 - accuracy: 0.2703
1/1 [==============================] - 0s 142ms/step - loss: 1.4135 - accuracy: 0.4146
4/4 [==============================] - 0s 4ms/step - loss: 1.5339 - accuracy: 0.3027
1/1 [==============================] - 0s 142ms/step - loss: 1.4548 - accuracy: 0.2927
4/4 [==============================] - 0s 4ms/step - loss: 1.5339 - accuracy: 0.2919
1/1 [==============================] - 0s 133ms/step - loss: 1.5774 - accuracy: 0.3171
4/4 [==============================] - 0s 5ms/step - loss: 1.6337 - accuracy: 0.1514
1/1 [==============================] - 0s 138ms/step - loss: 1.6671 - accuracy: 0.2439
4/4 [==============================] - 0s 5ms/step - loss: 1.5724 - accuracy: 0.2514
1/1 [==============================] - 0s 134ms/step - loss: 1.4730 - accuracy: 0.3415
4/4 [==============================] - 0s 6ms/step - loss: 1.6018 - accuracy: 0.2737
1/1 [==============================] - 0s 136ms/step - loss: 1.4574 - accuracy: 0.3571
4/4 [==============================] - 0s 5ms/step - loss: 1.5208 - accuracy: 0.2838
1/1 [==============================] - 0s 165ms/step - loss: 1.5456 - accuracy: 0.3415
4/4 [==============================] - 0s 5ms/step - loss: 1.5323 - accuracy: 0.3162
1/1 [==============================] - 0s 133ms/step - loss: 1.3542 - accuracy: 0.3902
4/4 [==============================] - 1s 8ms/step - loss: 1.5257 - accuracy: 0.3216
1/1 [==============================] - 0s 191ms/step - loss: 1.7185 - accuracy: 0.2195
4/4 [==============================] - 1s 8ms/step - loss: 1.5401 - accuracy: 0.2919
1/1 [==============================] - 0s 180ms/step - loss: 1.3324 - accuracy: 0.3415
4/4 [==============================] - 1s 9ms/step - loss: 1.5580 - accuracy: 0.2757
1/1 [==============================] - 0s 215ms/step - loss: 1.3493 - accuracy: 0.4390
4/4 [==============================] - 0s 5ms/step - loss: 1.5231 - accuracy: 0.3108
1/1 [==============================] - 0s 126ms/step - loss: 1.4108 - accuracy: 0.3171
4/4 [==============================] - 1s 6ms/step - loss: 1.5229 - accuracy: 0.2892
1/1 [==============================] - 0s 136ms/step - loss: 1.5686 - accuracy: 0.2927
4/4 [==============================] - 0s 5ms/step - loss: 1.5227 - accuracy: 0.3027
1/1 [==============================] - 0s 132ms/step - loss: 1.6940 - accuracy: 0.2927
4/4 [==============================] - 0s 5ms/step - loss: 1.5223 - accuracy: 0.3189
1/1 [==============================] - 0s 133ms/step - loss: 1.4527 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.6134 - accuracy: 0.2276
3/3 [==============================] - 0s 6ms/step - loss: 1.5449 - accuracy: 0.3095
19/19 [==============================] - 0s 2ms/step - loss: 1.5374 - accuracy: 0.3054
3/3 [==============================] - 0s 4ms/step - loss: 1.5639 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.7059 - accuracy: 0.0946
3/3 [==============================] - 0s 4ms/step - loss: 1.6475 - accuracy: 0.0732
19/19 [==============================] - 0s 2ms/step - loss: 1.6075 - accuracy: 0.2054
3/3 [==============================] - 0s 3ms/step - loss: 1.5751 - accuracy: 0.2439
19/19 [==============================] - 0s 2ms/step - loss: 1.5722 - accuracy: 0.2946
3/3 [==============================] - 0s 4ms/step - loss: 1.4362 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.6431 - accuracy: 0.1838
3/3 [==============================] - 0s 3ms/step - loss: 1.5025 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.5499 - accuracy: 0.2946
3/3 [==============================] - 0s 3ms/step - loss: 1.5466 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5399 - accuracy: 0.3351
3/3 [==============================] - 0s 3ms/step - loss: 1.5553 - accuracy: 0.3171
19/19 [==============================] - 1s 3ms/step - loss: 1.5598 - accuracy: 0.3000
3/3 [==============================] - 0s 3ms/step - loss: 1.6323 - accuracy: 0.2683
19/19 [==============================] - 1s 3ms/step - loss: 1.5927 - accuracy: 0.2459
3/3 [==============================] - 0s 6ms/step - loss: 1.5701 - accuracy: 0.1707
19/19 [==============================] - 1s 4ms/step - loss: 1.5439 - accuracy: 0.2466
3/3 [==============================] - 0s 3ms/step - loss: 1.4505 - accuracy: 0.4048
19/19 [==============================] - 0s 2ms/step - loss: 1.5638 - accuracy: 0.2892
3/3 [==============================] - 0s 3ms/step - loss: 1.5063 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.5026 - accuracy: 0.3162
3/3 [==============================] - 0s 4ms/step - loss: 1.3712 - accuracy: 0.4146
19/19 [==============================] - 0s 2ms/step - loss: 1.5193 - accuracy: 0.3189
3/3 [==============================] - 0s 3ms/step - loss: 1.5758 - accuracy: 0.2195
19/19 [==============================] - 0s 2ms/step - loss: 1.5259 - accuracy: 0.3027
3/3 [==============================] - 0s 3ms/step - loss: 1.3634 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5368 - accuracy: 0.2595
3/3 [==============================] - 0s 3ms/step - loss: 1.3417 - accuracy: 0.4390
19/19 [==============================] - 0s 2ms/step - loss: 1.5177 - accuracy: 0.3243
3/3 [==============================] - 0s 4ms/step - loss: 1.4145 - accuracy: 0.3171
19/19 [==============================] - 0s 3ms/step - loss: 1.5381 - accuracy: 0.3243
3/3 [==============================] - 0s 4ms/step - loss: 1.5585 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5227 - accuracy: 0.3162
3/3 [==============================] - 0s 4ms/step - loss: 1.7019 - accuracy: 0.2439
19/19 [==============================] - 0s 2ms/step - loss: 1.5445 - accuracy: 0.2946
3/3 [==============================] - 0s 3ms/step - loss: 1.4769 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5350 - accuracy: 0.2602
3/3 [==============================] - 0s 4ms/step - loss: 1.4632 - accuracy: 0.3333
19/19 [==============================] - 0s 2ms/step - loss: 1.4847 - accuracy: 0.3081
3/3 [==============================] - 0s 4ms/step - loss: 1.5086 - accuracy: 0.3902
19/19 [==============================] - 0s 2ms/step - loss: 1.5213 - accuracy: 0.3108
3/3 [==============================] - 0s 4ms/step - loss: 1.2641 - accuracy: 0.3902
19/19 [==============================] - 0s 2ms/step - loss: 1.4720 - accuracy: 0.3243
3/3 [==============================] - 0s 4ms/step - loss: 1.7523 - accuracy: 0.2195
19/19 [==============================] - 1s 3ms/step - loss: 1.5023 - accuracy: 0.2973
3/3 [==============================] - 0s 5ms/step - loss: 1.3599 - accuracy: 0.3171
19/19 [==============================] - 1s 4ms/step - loss: 1.4886 - accuracy: 0.3108
3/3 [==============================] - 0s 4ms/step - loss: 1.3114 - accuracy: 0.4390
19/19 [==============================] - 1s 2ms/step - loss: 1.5066 - accuracy: 0.3000
3/3 [==============================] - 0s 3ms/step - loss: 1.3558 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.4786 - accuracy: 0.3108
3/3 [==============================] - 0s 3ms/step - loss: 1.5794 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.4772 - accuracy: 0.3378
3/3 [==============================] - 0s 5ms/step - loss: 1.8615 - accuracy: 0.2927
19/19 [==============================] - 1s 2ms/step - loss: 1.5006 - accuracy: 0.3297
3/3 [==============================] - 0s 3ms/step - loss: 1.4393 - accuracy: 0.3902
19/19 [==============================] - 0s 3ms/step - loss: 1.5224 - accuracy: 0.2900
3/3 [==============================] - 0s 4ms/step - loss: 1.4481 - accuracy: 0.3571
19/19 [==============================] - 0s 3ms/step - loss: 1.4717 - accuracy: 0.3243
3/3 [==============================] - 0s 4ms/step - loss: 1.5549 - accuracy: 0.2683
19/19 [==============================] - 0s 3ms/step - loss: 1.4596 - accuracy: 0.3243
3/3 [==============================] - 0s 4ms/step - loss: 1.2891 - accuracy: 0.3659
19/19 [==============================] - 0s 3ms/step - loss: 1.4542 - accuracy: 0.3568
3/3 [==============================] - 1s 6ms/step - loss: 1.7417 - accuracy: 0.2683
19/19 [==============================] - 1s 6ms/step - loss: 1.4942 - accuracy: 0.2973
3/3 [==============================] - 1s 3ms/step - loss: 1.3097 - accuracy: 0.3171
19/19 [==============================] - 1s 5ms/step - loss: 1.4934 - accuracy: 0.2811
3/3 [==============================] - 0s 5ms/step - loss: 1.3123 - accuracy: 0.4634
19/19 [==============================] - 1s 6ms/step - loss: 1.4788 - accuracy: 0.3054
3/3 [==============================] - 0s 4ms/step - loss: 1.3539 - accuracy: 0.3659
19/19 [==============================] - 1s 5ms/step - loss: 1.4499 - accuracy: 0.3649
3/3 [==============================] - 0s 5ms/step - loss: 1.5354 - accuracy: 0.2683
19/19 [==============================] - 1s 5ms/step - loss: 1.4665 - accuracy: 0.3432
3/3 [==============================] - 0s 3ms/step - loss: 1.8889 - accuracy: 0.2195
19/19 [==============================] - 0s 3ms/step - loss: 1.4993 - accuracy: 0.3108
3/3 [==============================] - 0s 4ms/step - loss: 1.3959 - accuracy: 0.4146
19/19 [==============================] - 0s 2ms/step - loss: 1.6288 - accuracy: 0.1789
3/3 [==============================] - 0s 3ms/step - loss: 1.5790 - accuracy: 0.2381
19/19 [==============================] - 0s 2ms/step - loss: 1.5772 - accuracy: 0.3108
3/3 [==============================] - 0s 4ms/step - loss: 1.6163 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.6173 - accuracy: 0.1757
3/3 [==============================] - 0s 3ms/step - loss: 1.5150 - accuracy: 0.2439
19/19 [==============================] - 0s 2ms/step - loss: 1.5802 - accuracy: 0.2703
3/3 [==============================] - 0s 4ms/step - loss: 1.6244 - accuracy: 0.1951
19/19 [==============================] - 0s 2ms/step - loss: 1.5426 - accuracy: 0.2946
3/3 [==============================] - 0s 4ms/step - loss: 1.4645 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5416 - accuracy: 0.2946
3/3 [==============================] - 0s 4ms/step - loss: 1.4226 - accuracy: 0.3902
19/19 [==============================] - 0s 2ms/step - loss: 1.5572 - accuracy: 0.2676
3/3 [==============================] - 0s 4ms/step - loss: 1.4439 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.6062 - accuracy: 0.2243
3/3 [==============================] - 0s 4ms/step - loss: 1.5794 - accuracy: 0.2683
19/19 [==============================] - 0s 2ms/step - loss: 1.5669 - accuracy: 0.2811
3/3 [==============================] - 0s 3ms/step - loss: 1.6647 - accuracy: 0.2439
19/19 [==============================] - 0s 2ms/step - loss: 1.5598 - accuracy: 0.3189
3/3 [==============================] - 0s 3ms/step - loss: 1.5067 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.5994 - accuracy: 0.2547
3/3 [==============================] - 0s 3ms/step - loss: 1.4797 - accuracy: 0.3333
19/19 [==============================] - 0s 2ms/step - loss: 1.4998 - accuracy: 0.3162
3/3 [==============================] - 0s 4ms/step - loss: 1.5213 - accuracy: 0.3171
19/19 [==============================] - 1s 3ms/step - loss: 1.5973 - accuracy: 0.2595
3/3 [==============================] - 0s 6ms/step - loss: 1.3622 - accuracy: 0.3902
19/19 [==============================] - 1s 4ms/step - loss: 1.5463 - accuracy: 0.3108
3/3 [==============================] - 0s 5ms/step - loss: 1.6088 - accuracy: 0.2439
19/19 [==============================] - 1s 2ms/step - loss: 1.5460 - accuracy: 0.3108
3/3 [==============================] - 0s 4ms/step - loss: 1.3493 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.5172 - accuracy: 0.2892
3/3 [==============================] - 0s 2ms/step - loss: 1.3215 - accuracy: 0.4634
19/19 [==============================] - 0s 2ms/step - loss: 1.5297 - accuracy: 0.3081
3/3 [==============================] - 0s 4ms/step - loss: 1.4213 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.4993 - accuracy: 0.3568
3/3 [==============================] - 0s 4ms/step - loss: 1.5535 - accuracy: 0.2683
19/19 [==============================] - 0s 2ms/step - loss: 1.5656 - accuracy: 0.2459
3/3 [==============================] - 0s 4ms/step - loss: 1.6778 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.5195 - accuracy: 0.3189
3/3 [==============================] - 0s 3ms/step - loss: 1.4420 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5255 - accuracy: 0.2954
3/3 [==============================] - 0s 5ms/step - loss: 1.4606 - accuracy: 0.3571
19/19 [==============================] - 1s 3ms/step - loss: 1.4766 - accuracy: 0.2973
3/3 [==============================] - 0s 3ms/step - loss: 1.5892 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.5157 - accuracy: 0.3108
3/3 [==============================] - 0s 3ms/step - loss: 1.3517 - accuracy: 0.3659
19/19 [==============================] - 0s 3ms/step - loss: 1.4811 - accuracy: 0.3432
3/3 [==============================] - 0s 4ms/step - loss: 1.7483 - accuracy: 0.2683
19/19 [==============================] - 0s 3ms/step - loss: 1.4899 - accuracy: 0.3270
3/3 [==============================] - 0s 4ms/step - loss: 1.3130 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.5062 - accuracy: 0.2973
3/3 [==============================] - 0s 4ms/step - loss: 1.2695 - accuracy: 0.5366
19/19 [==============================] - 0s 3ms/step - loss: 1.4896 - accuracy: 0.3351
3/3 [==============================] - 0s 8ms/step - loss: 1.3893 - accuracy: 0.3415
19/19 [==============================] - 1s 4ms/step - loss: 1.4872 - accuracy: 0.3351
3/3 [==============================] - 0s 3ms/step - loss: 1.5519 - accuracy: 0.2927
19/19 [==============================] - 1s 4ms/step - loss: 1.4985 - accuracy: 0.3243
3/3 [==============================] - 0s 4ms/step - loss: 1.8597 - accuracy: 0.2927
19/19 [==============================] - 1s 2ms/step - loss: 1.5077 - accuracy: 0.3081
3/3 [==============================] - 0s 4ms/step - loss: 1.4192 - accuracy: 0.4146
19/19 [==============================] - 0s 4ms/step - loss: 1.5311 - accuracy: 0.2981
3/3 [==============================] - 0s 5ms/step - loss: 1.4597 - accuracy: 0.2857
19/19 [==============================] - 0s 3ms/step - loss: 1.4579 - accuracy: 0.3432
3/3 [==============================] - 0s 4ms/step - loss: 1.5287 - accuracy: 0.3415
19/19 [==============================] - 0s 3ms/step - loss: 1.4715 - accuracy: 0.3216
3/3 [==============================] - 0s 4ms/step - loss: 1.2904 - accuracy: 0.3171
19/19 [==============================] - 0s 3ms/step - loss: 1.4575 - accuracy: 0.3324
3/3 [==============================] - 0s 4ms/step - loss: 1.6747 - accuracy: 0.1951
19/19 [==============================] - 0s 3ms/step - loss: 1.4861 - accuracy: 0.3162
3/3 [==============================] - 0s 4ms/step - loss: 1.3264 - accuracy: 0.2927
19/19 [==============================] - 0s 3ms/step - loss: 1.4937 - accuracy: 0.2946
3/3 [==============================] - 0s 5ms/step - loss: 1.2702 - accuracy: 0.4634
19/19 [==============================] - 0s 3ms/step - loss: 1.4649 - accuracy: 0.3432
3/3 [==============================] - 0s 4ms/step - loss: 1.3615 - accuracy: 0.3902
19/19 [==============================] - 0s 3ms/step - loss: 1.4453 - accuracy: 0.3703
3/3 [==============================] - 0s 4ms/step - loss: 1.5593 - accuracy: 0.2927
19/19 [==============================] - 0s 3ms/step - loss: 1.4521 - accuracy: 0.3432
3/3 [==============================] - 0s 4ms/step - loss: 1.8100 - accuracy: 0.2683
19/19 [==============================] - 0s 3ms/step - loss: 1.4959 - accuracy: 0.3027
3/3 [==============================] - 0s 3ms/step - loss: 1.4015 - accuracy: 0.4390
8/8 [==============================] - 0s 3ms/step - loss: 1.5601 - accuracy: 0.2791
1/1 [==============================] - 0s 126ms/step - loss: 1.5350 - accuracy: 0.3571
8/8 [==============================] - 0s 3ms/step - loss: 1.6200 - accuracy: 0.2297
1/1 [==============================] - 0s 126ms/step - loss: 1.6274 - accuracy: 0.1951
8/8 [==============================] - 1s 4ms/step - loss: 1.6430 - accuracy: 0.1730
1/1 [==============================] - 0s 179ms/step - loss: 1.5717 - accuracy: 0.3171
8/8 [==============================] - 1s 4ms/step - loss: 1.5950 - accuracy: 0.2189
1/1 [==============================] - 0s 175ms/step - loss: 1.5622 - accuracy: 0.2927
8/8 [==============================] - 1s 4ms/step - loss: 1.5007 - accuracy: 0.3514
1/1 [==============================] - 0s 211ms/step - loss: 1.4199 - accuracy: 0.3902
8/8 [==============================] - 0s 2ms/step - loss: 1.6218 - accuracy: 0.2459
1/1 [==============================] - 0s 137ms/step - loss: 1.5110 - accuracy: 0.3415
8/8 [==============================] - 0s 2ms/step - loss: 1.6052 - accuracy: 0.2270
1/1 [==============================] - 0s 124ms/step - loss: 1.5615 - accuracy: 0.2927
8/8 [==============================] - 0s 2ms/step - loss: 1.5678 - accuracy: 0.2919
1/1 [==============================] - 0s 123ms/step - loss: 1.5779 - accuracy: 0.2439
8/8 [==============================] - 0s 3ms/step - loss: 1.6332 - accuracy: 0.2189
1/1 [==============================] - 0s 134ms/step - loss: 1.6172 - accuracy: 0.2683
8/8 [==============================] - 0s 2ms/step - loss: 1.5429 - accuracy: 0.3162
1/1 [==============================] - 0s 131ms/step - loss: 1.4790 - accuracy: 0.3902
8/8 [==============================] - 0s 3ms/step - loss: 1.5286 - accuracy: 0.3117
1/1 [==============================] - 0s 122ms/step - loss: 1.4704 - accuracy: 0.4048
8/8 [==============================] - 0s 3ms/step - loss: 1.5891 - accuracy: 0.2243
1/1 [==============================] - 0s 133ms/step - loss: 1.4897 - accuracy: 0.3902
8/8 [==============================] - 0s 3ms/step - loss: 1.5814 - accuracy: 0.2297
1/1 [==============================] - 0s 132ms/step - loss: 1.4548 - accuracy: 0.3171
8/8 [==============================] - 1s 3ms/step - loss: 1.5385 - accuracy: 0.3189
1/1 [==============================] - 0s 124ms/step - loss: 1.6513 - accuracy: 0.1951
8/8 [==============================] - 0s 3ms/step - loss: 1.5566 - accuracy: 0.2946
1/1 [==============================] - 0s 130ms/step - loss: 1.4521 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.5688 - accuracy: 0.2973
1/1 [==============================] - 0s 136ms/step - loss: 1.4507 - accuracy: 0.3902
8/8 [==============================] - 0s 3ms/step - loss: 1.5396 - accuracy: 0.3054
1/1 [==============================] - 0s 133ms/step - loss: 1.4510 - accuracy: 0.3171
8/8 [==============================] - 0s 3ms/step - loss: 1.5878 - accuracy: 0.2432
1/1 [==============================] - 0s 143ms/step - loss: 1.5644 - accuracy: 0.1951
8/8 [==============================] - 1s 4ms/step - loss: 1.5800 - accuracy: 0.3108
1/1 [==============================] - 0s 189ms/step - loss: 1.5936 - accuracy: 0.2683
8/8 [==============================] - 1s 4ms/step - loss: 1.5592 - accuracy: 0.2324
1/1 [==============================] - 0s 213ms/step - loss: 1.4603 - accuracy: 0.3659
8/8 [==============================] - 1s 5ms/step - loss: 1.5327 - accuracy: 0.3117
1/1 [==============================] - 0s 144ms/step - loss: 1.4494 - accuracy: 0.3571
8/8 [==============================] - 0s 4ms/step - loss: 1.4944 - accuracy: 0.3189
1/1 [==============================] - 0s 121ms/step - loss: 1.5773 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5607 - accuracy: 0.2730
1/1 [==============================] - 0s 126ms/step - loss: 1.3544 - accuracy: 0.3902
8/8 [==============================] - 0s 3ms/step - loss: 1.5621 - accuracy: 0.3027
1/1 [==============================] - 0s 125ms/step - loss: 1.6309 - accuracy: 0.2195
8/8 [==============================] - 0s 3ms/step - loss: 1.5216 - accuracy: 0.3054
1/1 [==============================] - 0s 137ms/step - loss: 1.3699 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5328 - accuracy: 0.2730
1/1 [==============================] - 0s 135ms/step - loss: 1.3533 - accuracy: 0.4146
8/8 [==============================] - 0s 3ms/step - loss: 1.5530 - accuracy: 0.3000
1/1 [==============================] - 0s 123ms/step - loss: 1.4192 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5073 - accuracy: 0.3270
1/1 [==============================] - 0s 123ms/step - loss: 1.5834 - accuracy: 0.3171
8/8 [==============================] - 0s 3ms/step - loss: 1.4919 - accuracy: 0.3459
1/1 [==============================] - 0s 124ms/step - loss: 1.6845 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.5481 - accuracy: 0.2919
1/1 [==============================] - 0s 131ms/step - loss: 1.4368 - accuracy: 0.3415
8/8 [==============================] - 0s 4ms/step - loss: 1.4907 - accuracy: 0.3306
1/1 [==============================] - 0s 124ms/step - loss: 1.4440 - accuracy: 0.3571
8/8 [==============================] - 0s 4ms/step - loss: 1.4924 - accuracy: 0.3162
1/1 [==============================] - 0s 134ms/step - loss: 1.5725 - accuracy: 0.3415
8/8 [==============================] - 0s 4ms/step - loss: 1.5126 - accuracy: 0.3216
1/1 [==============================] - 0s 136ms/step - loss: 1.3532 - accuracy: 0.3902
8/8 [==============================] - 0s 4ms/step - loss: 1.4977 - accuracy: 0.3108
1/1 [==============================] - 0s 128ms/step - loss: 1.7409 - accuracy: 0.2195
8/8 [==============================] - 1s 5ms/step - loss: 1.4872 - accuracy: 0.3000
1/1 [==============================] - 0s 185ms/step - loss: 1.3182 - accuracy: 0.3902
8/8 [==============================] - 1s 8ms/step - loss: 1.5054 - accuracy: 0.3054
1/1 [==============================] - 0s 192ms/step - loss: 1.3070 - accuracy: 0.4390
8/8 [==============================] - 1s 6ms/step - loss: 1.4856 - accuracy: 0.3378
1/1 [==============================] - 0s 192ms/step - loss: 1.3648 - accuracy: 0.3415
8/8 [==============================] - 0s 4ms/step - loss: 1.4992 - accuracy: 0.3324
1/1 [==============================] - 0s 139ms/step - loss: 1.6207 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.4824 - accuracy: 0.3189
1/1 [==============================] - 0s 122ms/step - loss: 1.8225 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.5151 - accuracy: 0.3081
1/1 [==============================] - 0s 130ms/step - loss: 1.3795 - accuracy: 0.3659
8/8 [==============================] - 0s 3ms/step - loss: 1.6086 - accuracy: 0.2331
1/1 [==============================] - 0s 133ms/step - loss: 1.5313 - accuracy: 0.2857
8/8 [==============================] - 0s 3ms/step - loss: 1.5541 - accuracy: 0.3054
1/1 [==============================] - 0s 123ms/step - loss: 1.4607 - accuracy: 0.3902
8/8 [==============================] - 0s 3ms/step - loss: 1.5506 - accuracy: 0.2919
1/1 [==============================] - 0s 120ms/step - loss: 1.5074 - accuracy: 0.3659
8/8 [==============================] - 0s 2ms/step - loss: 1.6402 - accuracy: 0.2162
1/1 [==============================] - 0s 132ms/step - loss: 1.5779 - accuracy: 0.2683
8/8 [==============================] - 0s 3ms/step - loss: 1.5743 - accuracy: 0.2757
1/1 [==============================] - 0s 133ms/step - loss: 1.4931 - accuracy: 0.3902
8/8 [==============================] - 0s 2ms/step - loss: 1.5522 - accuracy: 0.2919
1/1 [==============================] - 0s 124ms/step - loss: 1.5278 - accuracy: 0.2439
8/8 [==============================] - 1s 3ms/step - loss: 1.6044 - accuracy: 0.2297
1/1 [==============================] - 0s 132ms/step - loss: 1.5535 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5976 - accuracy: 0.2297
1/1 [==============================] - 0s 148ms/step - loss: 1.5678 - accuracy: 0.1951
8/8 [==============================] - 0s 3ms/step - loss: 1.5418 - accuracy: 0.2784
1/1 [==============================] - 0s 144ms/step - loss: 1.6211 - accuracy: 0.2683
8/8 [==============================] - 0s 3ms/step - loss: 1.6771 - accuracy: 0.2162
1/1 [==============================] - 0s 190ms/step - loss: 1.6128 - accuracy: 0.2927
8/8 [==============================] - 1s 5ms/step - loss: 1.5560 - accuracy: 0.2791
1/1 [==============================] - 0s 182ms/step - loss: 1.5071 - accuracy: 0.3571
8/8 [==============================] - 1s 4ms/step - loss: 1.5523 - accuracy: 0.3000
1/1 [==============================] - 0s 216ms/step - loss: 1.5586 - accuracy: 0.3415
8/8 [==============================] - 1s 3ms/step - loss: 1.5871 - accuracy: 0.2730
1/1 [==============================] - 0s 125ms/step - loss: 1.4407 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5889 - accuracy: 0.2649
1/1 [==============================] - 0s 137ms/step - loss: 1.6060 - accuracy: 0.2195
8/8 [==============================] - 0s 3ms/step - loss: 1.5942 - accuracy: 0.2324
1/1 [==============================] - 0s 131ms/step - loss: 1.4521 - accuracy: 0.3171
8/8 [==============================] - 0s 3ms/step - loss: 1.5755 - accuracy: 0.2514
1/1 [==============================] - 0s 124ms/step - loss: 1.4879 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.6234 - accuracy: 0.2351
1/1 [==============================] - 0s 125ms/step - loss: 1.5053 - accuracy: 0.4146
8/8 [==============================] - 0s 3ms/step - loss: 1.5522 - accuracy: 0.2892
1/1 [==============================] - 0s 132ms/step - loss: 1.5708 - accuracy: 0.2683
8/8 [==============================] - 0s 3ms/step - loss: 1.6093 - accuracy: 0.2757
1/1 [==============================] - 0s 122ms/step - loss: 1.6094 - accuracy: 0.1951
8/8 [==============================] - 0s 4ms/step - loss: 1.5437 - accuracy: 0.3324
1/1 [==============================] - 0s 128ms/step - loss: 1.4591 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5209 - accuracy: 0.3279
1/1 [==============================] - 0s 126ms/step - loss: 1.4527 - accuracy: 0.3571
8/8 [==============================] - 0s 3ms/step - loss: 1.4934 - accuracy: 0.3162
1/1 [==============================] - 0s 135ms/step - loss: 1.5705 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5523 - accuracy: 0.2865
1/1 [==============================] - 0s 125ms/step - loss: 1.3865 - accuracy: 0.3902
8/8 [==============================] - 0s 3ms/step - loss: 1.5117 - accuracy: 0.3054
1/1 [==============================] - 0s 149ms/step - loss: 1.7356 - accuracy: 0.2195
8/8 [==============================] - 0s 3ms/step - loss: 1.5382 - accuracy: 0.2919
1/1 [==============================] - 0s 130ms/step - loss: 1.3943 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5318 - accuracy: 0.2811
1/1 [==============================] - 0s 136ms/step - loss: 1.3649 - accuracy: 0.4390
8/8 [==============================] - 1s 6ms/step - loss: 1.5495 - accuracy: 0.3135
1/1 [==============================] - 0s 188ms/step - loss: 1.4064 - accuracy: 0.3171
8/8 [==============================] - 1s 5ms/step - loss: 1.4962 - accuracy: 0.3243
1/1 [==============================] - 0s 199ms/step - loss: 1.5302 - accuracy: 0.2927
8/8 [==============================] - 1s 5ms/step - loss: 1.5156 - accuracy: 0.3081
1/1 [==============================] - 0s 185ms/step - loss: 1.7602 - accuracy: 0.2927
8/8 [==============================] - 1s 3ms/step - loss: 1.5393 - accuracy: 0.2892
1/1 [==============================] - 0s 131ms/step - loss: 1.4013 - accuracy: 0.3902
8/8 [==============================] - 0s 4ms/step - loss: 1.5248 - accuracy: 0.2900
1/1 [==============================] - 0s 124ms/step - loss: 1.4205 - accuracy: 0.3571
8/8 [==============================] - 0s 4ms/step - loss: 1.5155 - accuracy: 0.3027
1/1 [==============================] - 0s 128ms/step - loss: 1.5869 - accuracy: 0.3415
8/8 [==============================] - 0s 4ms/step - loss: 1.5256 - accuracy: 0.2946
1/1 [==============================] - 0s 126ms/step - loss: 1.3590 - accuracy: 0.3902
8/8 [==============================] - 0s 5ms/step - loss: 1.5173 - accuracy: 0.3027
1/1 [==============================] - 0s 124ms/step - loss: 1.6901 - accuracy: 0.2195
8/8 [==============================] - 0s 4ms/step - loss: 1.5133 - accuracy: 0.2946
1/1 [==============================] - 0s 135ms/step - loss: 1.3030 - accuracy: 0.3415
8/8 [==============================] - 0s 4ms/step - loss: 1.5096 - accuracy: 0.3000
1/1 [==============================] - 0s 129ms/step - loss: 1.3184 - accuracy: 0.4146
8/8 [==============================] - 0s 4ms/step - loss: 1.5086 - accuracy: 0.3135
1/1 [==============================] - 0s 150ms/step - loss: 1.3771 - accuracy: 0.3171
8/8 [==============================] - 0s 4ms/step - loss: 1.4895 - accuracy: 0.3486
1/1 [==============================] - 0s 130ms/step - loss: 1.5176 - accuracy: 0.2927
8/8 [==============================] - 1s 6ms/step - loss: 1.4555 - accuracy: 0.3541
1/1 [==============================] - 0s 137ms/step - loss: 1.8275 - accuracy: 0.2927
8/8 [==============================] - 0s 4ms/step - loss: 1.5106 - accuracy: 0.3189
1/1 [==============================] - 0s 130ms/step - loss: 1.4319 - accuracy: 0.3659
21/21 [==============================] - 0s 2ms/step - loss: 1.5214 - accuracy: 0.2920
best parameters for ANN:  {'batch_size': 20, 'nb_epoch': 50, 'unit': 200}
best score for ANN: 0.3552264839410782

best parameters for ANN: {'batch_size': 20, 'nb_epoch': 50, 'unit': 200}

best score for ANN: 0.3552264839410782

def NN_Model_Tuned_CV(X_train, X_test, y_train, y_test):
  in_dim = X_train.shape[1]

  tf.random.set_seed(7)

  model = Sequential()
  model.add(Dense(100, activation='relu', input_dim = in_dim))
  model.add(Dropout(0.2))
  model.add(Dense(50, activation='relu'))
  model.add(Dense(50, activation='relu'))
  model.add(Dense(5 , activation='softmax'))

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=0, patience=3)

  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

  print(model.summary())

  history=model.fit(X_train, y_train_cat, validation_split=0.2, epochs = 20, batch_size = 50, callbacks=[early_stopping])

  train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  result_kfold_df= pd.DataFrame({'model': ['Neural Network'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })



  hist= pd.DataFrame(history.history)

  for col in hist.columns:
    print(col)
    plt.plot(hist[col])
    plt.plot(hist[col])
    plt.title('model-'+col)
    plt.ylabel(col)
    plt.xlabel('epoch')
    plt.show()

  return result_kfold_df

NN_Model_Tuned_CV(X_train_cv, X_test_cv, y_train_cv, y_test_cv)

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense (Dense)               (None, 100)               20100     
                                                                 
 dropout (Dropout)           (None, 100)               0         
                                                                 
 dense_1 (Dense)             (None, 50)                5050      
                                                                 
 dense_2 (Dense)             (None, 50)                2550      
                                                                 
 dense_3 (Dense)             (None, 5)                 255       
                                                                 
=================================================================
Total params: 27,955
Trainable params: 27,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
6/6 [==============================] - 1s 43ms/step - loss: 1.5864 - accuracy: 0.2672 - val_loss: 1.5705 - val_accuracy: 0.3182
Epoch 2/20
6/6 [==============================] - 0s 7ms/step - loss: 1.5059 - accuracy: 0.3511 - val_loss: 1.5491 - val_accuracy: 0.3030
Epoch 3/20
6/6 [==============================] - 0s 6ms/step - loss: 1.4504 - accuracy: 0.3511 - val_loss: 1.5307 - val_accuracy: 0.3030
Epoch 4/20
6/6 [==============================] - 0s 7ms/step - loss: 1.3891 - accuracy: 0.3588 - val_loss: 1.5202 - val_accuracy: 0.3030
Epoch 5/20
6/6 [==============================] - 0s 9ms/step - loss: 1.3224 - accuracy: 0.3931 - val_loss: 1.5172 - val_accuracy: 0.2879
Epoch 6/20
6/6 [==============================] - 0s 7ms/step - loss: 1.2864 - accuracy: 0.3931 - val_loss: 1.5150 - val_accuracy: 0.2727
Epoch 7/20
6/6 [==============================] - 0s 6ms/step - loss: 1.2191 - accuracy: 0.4733 - val_loss: 1.5133 - val_accuracy: 0.2424
Epoch 8/20
6/6 [==============================] - 0s 6ms/step - loss: 1.1442 - accuracy: 0.5267 - val_loss: 1.5133 - val_accuracy: 0.2273
Epoch 9/20
6/6 [==============================] - 0s 6ms/step - loss: 1.0879 - accuracy: 0.5878 - val_loss: 1.5031 - val_accuracy: 0.2121
Epoch 10/20
6/6 [==============================] - 0s 6ms/step - loss: 1.0212 - accuracy: 0.6489 - val_loss: 1.4910 - val_accuracy: 0.2424
Epoch 11/20
6/6 [==============================] - 0s 7ms/step - loss: 0.9452 - accuracy: 0.6870 - val_loss: 1.4807 - val_accuracy: 0.2576
Epoch 12/20
6/6 [==============================] - 0s 7ms/step - loss: 0.8687 - accuracy: 0.7061 - val_loss: 1.4806 - val_accuracy: 0.2879
Epoch 13/20
6/6 [==============================] - 0s 6ms/step - loss: 0.7890 - accuracy: 0.7290 - val_loss: 1.5038 - val_accuracy: 0.2727
Epoch 14/20
6/6 [==============================] - 0s 10ms/step - loss: 0.7201 - accuracy: 0.7824 - val_loss: 1.5429 - val_accuracy: 0.3030
Epoch 15/20
6/6 [==============================] - 0s 7ms/step - loss: 0.6335 - accuracy: 0.8511 - val_loss: 1.5787 - val_accuracy: 0.2424
11/11 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.777439	0.361446	0.779701	0.343573

NN_Model_Tuned_CV(X_train_cv_smote, X_test_cv, y_train_cv_smote, y_test_cv)

Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_4 (Dense)             (None, 100)               20100     
                                                                 
 dropout_1 (Dropout)         (None, 100)               0         
                                                                 
 dense_5 (Dense)             (None, 50)                5050      
                                                                 
 dense_6 (Dense)             (None, 50)                2550      
                                                                 
 dense_7 (Dense)             (None, 5)                 255       
                                                                 
=================================================================
Total params: 27,955
Trainable params: 27,955
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
9/9 [==============================] - 1s 40ms/step - loss: 1.5658 - accuracy: 0.2591 - val_loss: 1.7398 - val_accuracy: 0.0182
Epoch 2/20
9/9 [==============================] - 0s 9ms/step - loss: 1.4833 - accuracy: 0.4136 - val_loss: 1.8479 - val_accuracy: 0.0000e+00
Epoch 3/20
9/9 [==============================] - 0s 9ms/step - loss: 1.3979 - accuracy: 0.4841 - val_loss: 1.9199 - val_accuracy: 0.0000e+00
Epoch 4/20
9/9 [==============================] - 0s 11ms/step - loss: 1.2984 - accuracy: 0.5841 - val_loss: 1.9556 - val_accuracy: 0.0091
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.503636	0.373494	0.419851	0.290246

Tuned_ANN(X_cv_fullset, y_cv_fullset)

4/4 [==============================] - 1s 7ms/step - loss: 1.6018 - accuracy: 0.3035
1/1 [==============================] - 0s 420ms/step - loss: 1.5532 - accuracy: 0.4048
4/4 [==============================] - 1s 5ms/step - loss: 1.6785 - accuracy: 0.1514
1/1 [==============================] - 0s 225ms/step - loss: 1.6837 - accuracy: 0.1220
4/4 [==============================] - 1s 5ms/step - loss: 1.6579 - accuracy: 0.1973
1/1 [==============================] - 0s 189ms/step - loss: 1.6494 - accuracy: 0.2195
4/4 [==============================] - 1s 3ms/step - loss: 1.7069 - accuracy: 0.1568

WARNING:tensorflow:5 out of the last 16 calls to <function Model.make_test_function.<locals>.test_function at 0x7d4c7c27f520> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

1/1 [==============================] - 0s 129ms/step - loss: 1.6241 - accuracy: 0.1707
4/4 [==============================] - 0s 3ms/step - loss: 1.5988 - accuracy: 0.2243

WARNING:tensorflow:6 out of the last 17 calls to <function Model.make_test_function.<locals>.test_function at 0x7d4c7c3f3640> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

1/1 [==============================] - 0s 122ms/step - loss: 1.5457 - accuracy: 0.1951
4/4 [==============================] - 0s 3ms/step - loss: 1.6441 - accuracy: 0.1811
1/1 [==============================] - 0s 134ms/step - loss: 1.5959 - accuracy: 0.2683
4/4 [==============================] - 0s 4ms/step - loss: 1.6480 - accuracy: 0.1919
1/1 [==============================] - 0s 117ms/step - loss: 1.5358 - accuracy: 0.2927
4/4 [==============================] - 0s 3ms/step - loss: 1.7000 - accuracy: 0.1865
1/1 [==============================] - 0s 137ms/step - loss: 1.5966 - accuracy: 0.2683
4/4 [==============================] - 0s 3ms/step - loss: 1.5930 - accuracy: 0.2568
1/1 [==============================] - 0s 129ms/step - loss: 1.5196 - accuracy: 0.3415
4/4 [==============================] - 0s 3ms/step - loss: 1.5946 - accuracy: 0.2459
1/1 [==============================] - 0s 117ms/step - loss: 1.5760 - accuracy: 0.2439
4/4 [==============================] - 0s 4ms/step - loss: 1.5577 - accuracy: 0.3089
1/1 [==============================] - 0s 150ms/step - loss: 1.5068 - accuracy: 0.3810
4/4 [==============================] - 0s 4ms/step - loss: 1.6044 - accuracy: 0.2216
1/1 [==============================] - 0s 132ms/step - loss: 1.5745 - accuracy: 0.3415
4/4 [==============================] - 0s 3ms/step - loss: 1.6421 - accuracy: 0.1595
1/1 [==============================] - 0s 129ms/step - loss: 1.5915 - accuracy: 0.2683
4/4 [==============================] - 1s 6ms/step - loss: 1.6819 - accuracy: 0.1405
1/1 [==============================] - 0s 177ms/step - loss: 1.5442 - accuracy: 0.3659
4/4 [==============================] - 1s 5ms/step - loss: 1.5476 - accuracy: 0.3162
1/1 [==============================] - 1s 507ms/step - loss: 1.4527 - accuracy: 0.2927
4/4 [==============================] - 1s 4ms/step - loss: 1.5677 - accuracy: 0.2649
1/1 [==============================] - 0s 125ms/step - loss: 1.4521 - accuracy: 0.4146
4/4 [==============================] - 0s 3ms/step - loss: 1.5317 - accuracy: 0.2919
1/1 [==============================] - 0s 141ms/step - loss: 1.4291 - accuracy: 0.3415
4/4 [==============================] - 0s 4ms/step - loss: 1.5815 - accuracy: 0.2703
1/1 [==============================] - 0s 122ms/step - loss: 1.5430 - accuracy: 0.2439
4/4 [==============================] - 0s 4ms/step - loss: 1.5416 - accuracy: 0.3108
1/1 [==============================] - 0s 125ms/step - loss: 1.6511 - accuracy: 0.2927
4/4 [==============================] - 0s 4ms/step - loss: 1.6005 - accuracy: 0.2243
1/1 [==============================] - 0s 157ms/step - loss: 1.5157 - accuracy: 0.3415
4/4 [==============================] - 0s 4ms/step - loss: 1.5593 - accuracy: 0.2575
1/1 [==============================] - 0s 125ms/step - loss: 1.4594 - accuracy: 0.3333
4/4 [==============================] - 0s 4ms/step - loss: 1.5113 - accuracy: 0.3108
1/1 [==============================] - 0s 124ms/step - loss: 1.5279 - accuracy: 0.3415
4/4 [==============================] - 0s 5ms/step - loss: 1.6022 - accuracy: 0.2514
1/1 [==============================] - 0s 137ms/step - loss: 1.4361 - accuracy: 0.3415
4/4 [==============================] - 0s 4ms/step - loss: 1.5485 - accuracy: 0.3000
1/1 [==============================] - 0s 135ms/step - loss: 1.5858 - accuracy: 0.2195
4/4 [==============================] - 0s 4ms/step - loss: 1.5795 - accuracy: 0.2459
1/1 [==============================] - 0s 122ms/step - loss: 1.4194 - accuracy: 0.3415
4/4 [==============================] - 0s 4ms/step - loss: 1.5177 - accuracy: 0.3135
1/1 [==============================] - 0s 133ms/step - loss: 1.3267 - accuracy: 0.4390
4/4 [==============================] - 0s 4ms/step - loss: 1.5286 - accuracy: 0.3216
1/1 [==============================] - 0s 128ms/step - loss: 1.3945 - accuracy: 0.3902
4/4 [==============================] - 0s 4ms/step - loss: 1.5354 - accuracy: 0.2919
1/1 [==============================] - 0s 127ms/step - loss: 1.5140 - accuracy: 0.2927
4/4 [==============================] - 0s 4ms/step - loss: 1.4999 - accuracy: 0.3486
1/1 [==============================] - 0s 136ms/step - loss: 1.6909 - accuracy: 0.2439
4/4 [==============================] - 0s 4ms/step - loss: 1.6039 - accuracy: 0.2270
1/1 [==============================] - 0s 127ms/step - loss: 1.4964 - accuracy: 0.4390
4/4 [==============================] - 1s 6ms/step - loss: 1.5388 - accuracy: 0.2981
1/1 [==============================] - 0s 186ms/step - loss: 1.4470 - accuracy: 0.3571
4/4 [==============================] - 1s 7ms/step - loss: 1.5462 - accuracy: 0.2892
1/1 [==============================] - 0s 191ms/step - loss: 1.5226 - accuracy: 0.3415
4/4 [==============================] - 1s 7ms/step - loss: 1.5359 - accuracy: 0.2919
1/1 [==============================] - 0s 217ms/step - loss: 1.3280 - accuracy: 0.4146
4/4 [==============================] - 0s 5ms/step - loss: 1.5680 - accuracy: 0.2486
1/1 [==============================] - 1s 821ms/step - loss: 1.6484 - accuracy: 0.2195
4/4 [==============================] - 0s 5ms/step - loss: 1.5498 - accuracy: 0.2865
1/1 [==============================] - 0s 137ms/step - loss: 1.3725 - accuracy: 0.3415
4/4 [==============================] - 0s 5ms/step - loss: 1.5550 - accuracy: 0.2838
1/1 [==============================] - 0s 131ms/step - loss: 1.3501 - accuracy: 0.4634
4/4 [==============================] - 0s 5ms/step - loss: 1.5506 - accuracy: 0.2568
1/1 [==============================] - 0s 142ms/step - loss: 1.3846 - accuracy: 0.3171
4/4 [==============================] - 0s 5ms/step - loss: 1.5208 - accuracy: 0.3324
1/1 [==============================] - 0s 124ms/step - loss: 1.5666 - accuracy: 0.2927
4/4 [==============================] - 0s 4ms/step - loss: 1.5078 - accuracy: 0.3459
1/1 [==============================] - 0s 128ms/step - loss: 1.7095 - accuracy: 0.2927
4/4 [==============================] - 0s 5ms/step - loss: 1.5072 - accuracy: 0.3378
1/1 [==============================] - 0s 128ms/step - loss: 1.4667 - accuracy: 0.3415
4/4 [==============================] - 0s 3ms/step - loss: 1.6431 - accuracy: 0.1653
1/1 [==============================] - 0s 129ms/step - loss: 1.5714 - accuracy: 0.4048
4/4 [==============================] - 0s 3ms/step - loss: 1.5940 - accuracy: 0.2432
1/1 [==============================] - 0s 126ms/step - loss: 1.6070 - accuracy: 0.2683
4/4 [==============================] - 0s 3ms/step - loss: 1.6686 - accuracy: 0.1730
1/1 [==============================] - 0s 131ms/step - loss: 1.6391 - accuracy: 0.1463
4/4 [==============================] - 0s 3ms/step - loss: 1.6585 - accuracy: 0.1568
1/1 [==============================] - 0s 127ms/step - loss: 1.6192 - accuracy: 0.1951
4/4 [==============================] - 0s 3ms/step - loss: 1.6262 - accuracy: 0.1946
1/1 [==============================] - 0s 131ms/step - loss: 1.5765 - accuracy: 0.3171
4/4 [==============================] - 0s 3ms/step - loss: 1.6784 - accuracy: 0.1081
1/1 [==============================] - 0s 135ms/step - loss: 1.6826 - accuracy: 0.1220
4/4 [==============================] - 1s 5ms/step - loss: 1.6342 - accuracy: 0.2514
1/1 [==============================] - 0s 185ms/step - loss: 1.6144 - accuracy: 0.3171
4/4 [==============================] - 1s 5ms/step - loss: 1.5660 - accuracy: 0.3162
1/1 [==============================] - 0s 167ms/step - loss: 1.5489 - accuracy: 0.3171
4/4 [==============================] - 1s 6ms/step - loss: 1.6072 - accuracy: 0.2351
1/1 [==============================] - 0s 193ms/step - loss: 1.5990 - accuracy: 0.2195
4/4 [==============================] - 1s 3ms/step - loss: 1.5981 - accuracy: 0.2919
1/1 [==============================] - 0s 127ms/step - loss: 1.4889 - accuracy: 0.4390
4/4 [==============================] - 0s 4ms/step - loss: 1.6347 - accuracy: 0.1816
1/1 [==============================] - 0s 123ms/step - loss: 1.5550 - accuracy: 0.3095
4/4 [==============================] - 0s 3ms/step - loss: 1.5495 - accuracy: 0.3243
1/1 [==============================] - 0s 130ms/step - loss: 1.5847 - accuracy: 0.3415
4/4 [==============================] - 0s 4ms/step - loss: 1.5623 - accuracy: 0.3081
1/1 [==============================] - 0s 122ms/step - loss: 1.4418 - accuracy: 0.3659
4/4 [==============================] - 0s 3ms/step - loss: 1.5792 - accuracy: 0.2486
1/1 [==============================] - 0s 122ms/step - loss: 1.6110 - accuracy: 0.2439
4/4 [==============================] - 0s 4ms/step - loss: 1.6059 - accuracy: 0.2216
1/1 [==============================] - 0s 121ms/step - loss: 1.4461 - accuracy: 0.5366
4/4 [==============================] - 0s 4ms/step - loss: 1.5998 - accuracy: 0.2676
1/1 [==============================] - 0s 126ms/step - loss: 1.5211 - accuracy: 0.2927
4/4 [==============================] - 0s 5ms/step - loss: 1.5916 - accuracy: 0.2676
1/1 [==============================] - 0s 134ms/step - loss: 1.5357 - accuracy: 0.2927
4/4 [==============================] - 0s 4ms/step - loss: 1.5876 - accuracy: 0.2054
1/1 [==============================] - 0s 124ms/step - loss: 1.5318 - accuracy: 0.3902
4/4 [==============================] - 0s 4ms/step - loss: 1.5079 - accuracy: 0.3351
1/1 [==============================] - 0s 121ms/step - loss: 1.5759 - accuracy: 0.2927
4/4 [==============================] - 0s 7ms/step - loss: 1.5865 - accuracy: 0.3027
1/1 [==============================] - 0s 123ms/step - loss: 1.4814 - accuracy: 0.3659
4/4 [==============================] - 0s 4ms/step - loss: 1.5244 - accuracy: 0.3008
1/1 [==============================] - 0s 126ms/step - loss: 1.4536 - accuracy: 0.3571
4/4 [==============================] - 0s 4ms/step - loss: 1.5444 - accuracy: 0.3108
1/1 [==============================] - 0s 124ms/step - loss: 1.5507 - accuracy: 0.3415
4/4 [==============================] - 0s 5ms/step - loss: 1.6467 - accuracy: 0.2135
1/1 [==============================] - 0s 119ms/step - loss: 1.4719 - accuracy: 0.3902
4/4 [==============================] - 0s 4ms/step - loss: 1.5707 - accuracy: 0.2622
1/1 [==============================] - 0s 147ms/step - loss: 1.5865 - accuracy: 0.2439
4/4 [==============================] - 1s 7ms/step - loss: 1.6107 - accuracy: 0.2081
1/1 [==============================] - 0s 190ms/step - loss: 1.4414 - accuracy: 0.2927
4/4 [==============================] - 1s 7ms/step - loss: 1.5316 - accuracy: 0.3000
1/1 [==============================] - 1s 1s/step - loss: 1.3305 - accuracy: 0.4390
4/4 [==============================] - 0s 4ms/step - loss: 1.5784 - accuracy: 0.2730
1/1 [==============================] - 0s 123ms/step - loss: 1.4268 - accuracy: 0.3171
4/4 [==============================] - 0s 4ms/step - loss: 1.5235 - accuracy: 0.2892
1/1 [==============================] - 0s 117ms/step - loss: 1.5347 - accuracy: 0.2927
4/4 [==============================] - 0s 4ms/step - loss: 1.5287 - accuracy: 0.3108
1/1 [==============================] - 0s 132ms/step - loss: 1.7486 - accuracy: 0.2927
4/4 [==============================] - 0s 4ms/step - loss: 1.5735 - accuracy: 0.2568
1/1 [==============================] - 0s 122ms/step - loss: 1.4657 - accuracy: 0.3659
4/4 [==============================] - 0s 5ms/step - loss: 1.5657 - accuracy: 0.2602
1/1 [==============================] - 0s 127ms/step - loss: 1.4688 - accuracy: 0.3810
4/4 [==============================] - 0s 5ms/step - loss: 1.5023 - accuracy: 0.3459
1/1 [==============================] - 0s 125ms/step - loss: 1.5464 - accuracy: 0.2927
4/4 [==============================] - 0s 5ms/step - loss: 1.5443 - accuracy: 0.2649
1/1 [==============================] - 0s 137ms/step - loss: 1.3231 - accuracy: 0.3902
4/4 [==============================] - 0s 5ms/step - loss: 1.5375 - accuracy: 0.2649
1/1 [==============================] - 0s 132ms/step - loss: 1.6689 - accuracy: 0.2195
4/4 [==============================] - 0s 5ms/step - loss: 1.4905 - accuracy: 0.3378
1/1 [==============================] - 0s 127ms/step - loss: 1.3321 - accuracy: 0.3659
4/4 [==============================] - 0s 5ms/step - loss: 1.5188 - accuracy: 0.3189
1/1 [==============================] - 0s 123ms/step - loss: 1.3472 - accuracy: 0.4390
4/4 [==============================] - 0s 5ms/step - loss: 1.5058 - accuracy: 0.3297
1/1 [==============================] - 0s 133ms/step - loss: 1.4156 - accuracy: 0.3171
4/4 [==============================] - 0s 5ms/step - loss: 1.5621 - accuracy: 0.2703
1/1 [==============================] - 0s 130ms/step - loss: 1.5404 - accuracy: 0.2683
4/4 [==============================] - 0s 5ms/step - loss: 1.5494 - accuracy: 0.3108
1/1 [==============================] - 0s 123ms/step - loss: 1.7533 - accuracy: 0.2927
4/4 [==============================] - 0s 5ms/step - loss: 1.5334 - accuracy: 0.3135
1/1 [==============================] - 0s 132ms/step - loss: 1.4217 - accuracy: 0.4146
19/19 [==============================] - 1s 3ms/step - loss: 1.6073 - accuracy: 0.2114
3/3 [==============================] - 0s 2ms/step - loss: 1.4880 - accuracy: 0.4286
19/19 [==============================] - 1s 3ms/step - loss: 1.5519 - accuracy: 0.2838
3/3 [==============================] - 0s 3ms/step - loss: 1.5346 - accuracy: 0.3415
19/19 [==============================] - 1s 3ms/step - loss: 1.5178 - accuracy: 0.3351
3/3 [==============================] - 0s 4ms/step - loss: 1.4228 - accuracy: 0.3171
19/19 [==============================] - 1s 2ms/step - loss: 1.6154 - accuracy: 0.2270
3/3 [==============================] - 0s 3ms/step - loss: 1.5270 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5893 - accuracy: 0.2459
3/3 [==============================] - 0s 3ms/step - loss: 1.4706 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.5811 - accuracy: 0.2784
3/3 [==============================] - 0s 3ms/step - loss: 1.5069 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.5731 - accuracy: 0.2919
3/3 [==============================] - 0s 3ms/step - loss: 1.4767 - accuracy: 0.2683
19/19 [==============================] - 0s 2ms/step - loss: 1.5966 - accuracy: 0.2054
3/3 [==============================] - 0s 3ms/step - loss: 1.6232 - accuracy: 0.1707
19/19 [==============================] - 0s 2ms/step - loss: 1.5282 - accuracy: 0.2892
3/3 [==============================] - 0s 3ms/step - loss: 1.5977 - accuracy: 0.2683
19/19 [==============================] - 0s 2ms/step - loss: 1.5859 - accuracy: 0.2784
3/3 [==============================] - 0s 3ms/step - loss: 1.4767 - accuracy: 0.4634
19/19 [==============================] - 0s 2ms/step - loss: 1.5075 - accuracy: 0.3035
3/3 [==============================] - 0s 3ms/step - loss: 1.4304 - accuracy: 0.3571
19/19 [==============================] - 0s 3ms/step - loss: 1.5490 - accuracy: 0.2811
3/3 [==============================] - 0s 3ms/step - loss: 1.5163 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.5510 - accuracy: 0.2946
3/3 [==============================] - 0s 3ms/step - loss: 1.3551 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.4946 - accuracy: 0.3108
3/3 [==============================] - 0s 3ms/step - loss: 1.6633 - accuracy: 0.2683
19/19 [==============================] - 0s 2ms/step - loss: 1.5477 - accuracy: 0.3027
3/3 [==============================] - 0s 3ms/step - loss: 1.3159 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.5256 - accuracy: 0.2703
3/3 [==============================] - 0s 3ms/step - loss: 1.3112 - accuracy: 0.4878
19/19 [==============================] - 0s 2ms/step - loss: 1.5141 - accuracy: 0.3270
3/3 [==============================] - 0s 3ms/step - loss: 1.3580 - accuracy: 0.3171
19/19 [==============================] - 1s 3ms/step - loss: 1.5170 - accuracy: 0.3270
3/3 [==============================] - 0s 5ms/step - loss: 1.5263 - accuracy: 0.3171
19/19 [==============================] - 2s 4ms/step - loss: 1.5216 - accuracy: 0.3297
3/3 [==============================] - 0s 3ms/step - loss: 1.7090 - accuracy: 0.2683
19/19 [==============================] - 0s 2ms/step - loss: 1.5186 - accuracy: 0.3081
3/3 [==============================] - 0s 3ms/step - loss: 1.4186 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5118 - accuracy: 0.2818
3/3 [==============================] - 0s 3ms/step - loss: 1.4196 - accuracy: 0.3333
19/19 [==============================] - 0s 2ms/step - loss: 1.4583 - accuracy: 0.3324
3/3 [==============================] - 0s 3ms/step - loss: 1.4724 - accuracy: 0.2439
19/19 [==============================] - 0s 2ms/step - loss: 1.4723 - accuracy: 0.3162
3/3 [==============================] - 0s 4ms/step - loss: 1.2793 - accuracy: 0.4146
19/19 [==============================] - 0s 2ms/step - loss: 1.4637 - accuracy: 0.3351
3/3 [==============================] - 0s 3ms/step - loss: 1.6757 - accuracy: 0.2439
19/19 [==============================] - 0s 2ms/step - loss: 1.4886 - accuracy: 0.3270
3/3 [==============================] - 0s 3ms/step - loss: 1.3171 - accuracy: 0.3415
19/19 [==============================] - 0s 3ms/step - loss: 1.4710 - accuracy: 0.3135
3/3 [==============================] - 0s 3ms/step - loss: 1.2722 - accuracy: 0.5122
19/19 [==============================] - 0s 2ms/step - loss: 1.4737 - accuracy: 0.3162
3/3 [==============================] - 0s 3ms/step - loss: 1.3239 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.4813 - accuracy: 0.3108
3/3 [==============================] - 0s 3ms/step - loss: 1.5557 - accuracy: 0.2439
19/19 [==============================] - 0s 2ms/step - loss: 1.4755 - accuracy: 0.3189
3/3 [==============================] - 0s 3ms/step - loss: 1.7695 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5114 - accuracy: 0.3000
3/3 [==============================] - 0s 3ms/step - loss: 1.3950 - accuracy: 0.3902
19/19 [==============================] - 0s 3ms/step - loss: 1.5042 - accuracy: 0.3035
3/3 [==============================] - 0s 3ms/step - loss: 1.4331 - accuracy: 0.3810
19/19 [==============================] - 0s 3ms/step - loss: 1.4470 - accuracy: 0.3378
3/3 [==============================] - 0s 3ms/step - loss: 1.4476 - accuracy: 0.2683
19/19 [==============================] - 0s 5ms/step - loss: 1.4641 - accuracy: 0.3297
3/3 [==============================] - 0s 6ms/step - loss: 1.2871 - accuracy: 0.3659
19/19 [==============================] - 1s 5ms/step - loss: 1.4408 - accuracy: 0.3541
3/3 [==============================] - 0s 3ms/step - loss: 1.6358 - accuracy: 0.2683
19/19 [==============================] - 1s 5ms/step - loss: 1.4585 - accuracy: 0.3324
3/3 [==============================] - 0s 6ms/step - loss: 1.2720 - accuracy: 0.3171
19/19 [==============================] - 1s 3ms/step - loss: 1.4709 - accuracy: 0.2838
3/3 [==============================] - 0s 3ms/step - loss: 1.2602 - accuracy: 0.4146
19/19 [==============================] - 0s 3ms/step - loss: 1.4576 - accuracy: 0.3405
3/3 [==============================] - 0s 4ms/step - loss: 1.3325 - accuracy: 0.3171
19/19 [==============================] - 0s 3ms/step - loss: 1.4292 - accuracy: 0.3459
3/3 [==============================] - 0s 3ms/step - loss: 1.5439 - accuracy: 0.3171
19/19 [==============================] - 0s 3ms/step - loss: 1.4499 - accuracy: 0.3432
3/3 [==============================] - 0s 4ms/step - loss: 1.7478 - accuracy: 0.2683
19/19 [==============================] - 0s 3ms/step - loss: 1.4819 - accuracy: 0.3270
3/3 [==============================] - 0s 4ms/step - loss: 1.4131 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.6300 - accuracy: 0.2276
3/3 [==============================] - 0s 3ms/step - loss: 1.5507 - accuracy: 0.3810
19/19 [==============================] - 0s 2ms/step - loss: 1.6019 - accuracy: 0.2216
3/3 [==============================] - 0s 3ms/step - loss: 1.5439 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5642 - accuracy: 0.2676
3/3 [==============================] - 0s 3ms/step - loss: 1.4562 - accuracy: 0.3902
19/19 [==============================] - 0s 2ms/step - loss: 1.5795 - accuracy: 0.2676
3/3 [==============================] - 0s 3ms/step - loss: 1.6369 - accuracy: 0.1951
19/19 [==============================] - 0s 2ms/step - loss: 1.6195 - accuracy: 0.2162
3/3 [==============================] - 0s 4ms/step - loss: 1.5520 - accuracy: 0.2683
19/19 [==============================] - 0s 2ms/step - loss: 1.6020 - accuracy: 0.2000
3/3 [==============================] - 0s 3ms/step - loss: 1.4755 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5870 - accuracy: 0.2541
3/3 [==============================] - 0s 3ms/step - loss: 1.5243 - accuracy: 0.2195
19/19 [==============================] - 0s 2ms/step - loss: 1.5844 - accuracy: 0.2595
3/3 [==============================] - 0s 3ms/step - loss: 1.5818 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5004 - accuracy: 0.2946
3/3 [==============================] - 0s 3ms/step - loss: 1.6095 - accuracy: 0.3415
19/19 [==============================] - 1s 3ms/step - loss: 1.5950 - accuracy: 0.2676
3/3 [==============================] - 0s 3ms/step - loss: 1.4491 - accuracy: 0.4146
19/19 [==============================] - 2s 4ms/step - loss: 1.5212 - accuracy: 0.3279
3/3 [==============================] - 0s 4ms/step - loss: 1.4833 - accuracy: 0.3095
19/19 [==============================] - 0s 2ms/step - loss: 1.5209 - accuracy: 0.3162
3/3 [==============================] - 0s 3ms/step - loss: 1.5395 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5033 - accuracy: 0.3135
3/3 [==============================] - 0s 3ms/step - loss: 1.3183 - accuracy: 0.3902
19/19 [==============================] - 0s 2ms/step - loss: 1.4837 - accuracy: 0.3405
3/3 [==============================] - 0s 3ms/step - loss: 1.7276 - accuracy: 0.2195
19/19 [==============================] - 0s 2ms/step - loss: 1.5202 - accuracy: 0.2946
3/3 [==============================] - 0s 3ms/step - loss: 1.3408 - accuracy: 0.2683
19/19 [==============================] - 0s 2ms/step - loss: 1.5347 - accuracy: 0.2919
3/3 [==============================] - 0s 3ms/step - loss: 1.3070 - accuracy: 0.4146
19/19 [==============================] - 0s 2ms/step - loss: 1.5215 - accuracy: 0.3135
3/3 [==============================] - 0s 3ms/step - loss: 1.4389 - accuracy: 0.3415
19/19 [==============================] - 1s 3ms/step - loss: 1.5161 - accuracy: 0.2973
3/3 [==============================] - 0s 3ms/step - loss: 1.5833 - accuracy: 0.2683
19/19 [==============================] - 1s 4ms/step - loss: 1.4956 - accuracy: 0.3162
3/3 [==============================] - 0s 6ms/step - loss: 1.6268 - accuracy: 0.2683
19/19 [==============================] - 1s 2ms/step - loss: 1.5563 - accuracy: 0.3027
3/3 [==============================] - 0s 3ms/step - loss: 1.4600 - accuracy: 0.3902
19/19 [==============================] - 0s 2ms/step - loss: 1.5174 - accuracy: 0.2846
3/3 [==============================] - 0s 3ms/step - loss: 1.4417 - accuracy: 0.3810
19/19 [==============================] - 0s 2ms/step - loss: 1.4925 - accuracy: 0.2838
3/3 [==============================] - 0s 3ms/step - loss: 1.5384 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.4660 - accuracy: 0.3514
3/3 [==============================] - 0s 3ms/step - loss: 1.2653 - accuracy: 0.3902
19/19 [==============================] - 0s 3ms/step - loss: 1.4512 - accuracy: 0.3459
3/3 [==============================] - 0s 3ms/step - loss: 1.6655 - accuracy: 0.2439
19/19 [==============================] - 1s 4ms/step - loss: 1.5239 - accuracy: 0.2973
3/3 [==============================] - 0s 6ms/step - loss: 1.3057 - accuracy: 0.3659
19/19 [==============================] - 1s 4ms/step - loss: 1.4685 - accuracy: 0.3432
3/3 [==============================] - 0s 3ms/step - loss: 1.2923 - accuracy: 0.3659
19/19 [==============================] - 1s 2ms/step - loss: 1.4824 - accuracy: 0.2973
3/3 [==============================] - 0s 3ms/step - loss: 1.3480 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.4673 - accuracy: 0.3378
3/3 [==============================] - 0s 3ms/step - loss: 1.5393 - accuracy: 0.2439
19/19 [==============================] - 0s 2ms/step - loss: 1.4787 - accuracy: 0.3541
3/3 [==============================] - 0s 3ms/step - loss: 1.7362 - accuracy: 0.2683
19/19 [==============================] - 0s 2ms/step - loss: 1.5024 - accuracy: 0.2838
3/3 [==============================] - 0s 3ms/step - loss: 1.4182 - accuracy: 0.2927
19/19 [==============================] - 0s 3ms/step - loss: 1.4923 - accuracy: 0.2981
3/3 [==============================] - 0s 3ms/step - loss: 1.4695 - accuracy: 0.2857
19/19 [==============================] - 0s 3ms/step - loss: 1.4415 - accuracy: 0.3622
3/3 [==============================] - 0s 3ms/step - loss: 1.4491 - accuracy: 0.2683
19/19 [==============================] - 0s 3ms/step - loss: 1.4533 - accuracy: 0.3378
3/3 [==============================] - 0s 4ms/step - loss: 1.2472 - accuracy: 0.3171
19/19 [==============================] - 0s 3ms/step - loss: 1.4377 - accuracy: 0.3568
3/3 [==============================] - 0s 4ms/step - loss: 1.5960 - accuracy: 0.2927
19/19 [==============================] - 0s 3ms/step - loss: 1.4569 - accuracy: 0.3324
3/3 [==============================] - 0s 3ms/step - loss: 1.2801 - accuracy: 0.3659
19/19 [==============================] - 0s 3ms/step - loss: 1.4605 - accuracy: 0.3000
3/3 [==============================] - 0s 4ms/step - loss: 1.2702 - accuracy: 0.4634
19/19 [==============================] - 0s 3ms/step - loss: 1.4406 - accuracy: 0.3432
3/3 [==============================] - 0s 4ms/step - loss: 1.3077 - accuracy: 0.3415
19/19 [==============================] - 0s 3ms/step - loss: 1.4270 - accuracy: 0.3703
3/3 [==============================] - 0s 4ms/step - loss: 1.5201 - accuracy: 0.3171
19/19 [==============================] - 0s 3ms/step - loss: 1.4441 - accuracy: 0.3432
3/3 [==============================] - 0s 3ms/step - loss: 1.7438 - accuracy: 0.3171
19/19 [==============================] - 0s 3ms/step - loss: 1.4917 - accuracy: 0.2973
3/3 [==============================] - 0s 5ms/step - loss: 1.3676 - accuracy: 0.3659
8/8 [==============================] - 1s 3ms/step - loss: 1.6466 - accuracy: 0.1816
1/1 [==============================] - 0s 177ms/step - loss: 1.6424 - accuracy: 0.1905
8/8 [==============================] - 1s 4ms/step - loss: 1.5957 - accuracy: 0.2378
1/1 [==============================] - 0s 199ms/step - loss: 1.6062 - accuracy: 0.1463
8/8 [==============================] - 1s 3ms/step - loss: 1.6197 - accuracy: 0.2486
1/1 [==============================] - 0s 128ms/step - loss: 1.5987 - accuracy: 0.2439
8/8 [==============================] - 0s 2ms/step - loss: 1.5310 - accuracy: 0.3297
1/1 [==============================] - 0s 128ms/step - loss: 1.6256 - accuracy: 0.2439
8/8 [==============================] - 0s 2ms/step - loss: 1.5863 - accuracy: 0.2676
1/1 [==============================] - 0s 127ms/step - loss: 1.5255 - accuracy: 0.2927
8/8 [==============================] - 0s 2ms/step - loss: 1.6894 - accuracy: 0.1162
1/1 [==============================] - 0s 126ms/step - loss: 1.6255 - accuracy: 0.1707
8/8 [==============================] - 0s 3ms/step - loss: 1.6769 - accuracy: 0.2243
1/1 [==============================] - 0s 141ms/step - loss: 1.5634 - accuracy: 0.4878
8/8 [==============================] - 0s 3ms/step - loss: 1.6380 - accuracy: 0.1432
1/1 [==============================] - 0s 132ms/step - loss: 1.5943 - accuracy: 0.2195
8/8 [==============================] - 0s 3ms/step - loss: 1.5488 - accuracy: 0.2973
1/1 [==============================] - 0s 122ms/step - loss: 1.6783 - accuracy: 0.2683
8/8 [==============================] - 0s 2ms/step - loss: 1.6186 - accuracy: 0.2027
1/1 [==============================] - 0s 132ms/step - loss: 1.5781 - accuracy: 0.1951
8/8 [==============================] - 0s 3ms/step - loss: 1.5595 - accuracy: 0.2737
1/1 [==============================] - 0s 139ms/step - loss: 1.4258 - accuracy: 0.3571
8/8 [==============================] - 0s 3ms/step - loss: 1.5494 - accuracy: 0.3297
1/1 [==============================] - 0s 123ms/step - loss: 1.5580 - accuracy: 0.3171
8/8 [==============================] - 0s 3ms/step - loss: 1.5690 - accuracy: 0.2432
1/1 [==============================] - 0s 129ms/step - loss: 1.4223 - accuracy: 0.4146
8/8 [==============================] - 0s 3ms/step - loss: 1.5756 - accuracy: 0.2730
1/1 [==============================] - 0s 135ms/step - loss: 1.6007 - accuracy: 0.2195
8/8 [==============================] - 0s 3ms/step - loss: 1.6459 - accuracy: 0.2108
1/1 [==============================] - 0s 124ms/step - loss: 1.4940 - accuracy: 0.4390
8/8 [==============================] - 0s 3ms/step - loss: 1.5639 - accuracy: 0.2838
1/1 [==============================] - 0s 180ms/step - loss: 1.4127 - accuracy: 0.4878
8/8 [==============================] - 1s 5ms/step - loss: 1.5376 - accuracy: 0.2946
1/1 [==============================] - 0s 161ms/step - loss: 1.4343 - accuracy: 0.2683
8/8 [==============================] - 1s 4ms/step - loss: 1.5074 - accuracy: 0.3081
1/1 [==============================] - 0s 188ms/step - loss: 1.5571 - accuracy: 0.2927
8/8 [==============================] - 1s 5ms/step - loss: 1.5601 - accuracy: 0.2703
1/1 [==============================] - 0s 138ms/step - loss: 1.6141 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.5461 - accuracy: 0.2784
1/1 [==============================] - 0s 133ms/step - loss: 1.4628 - accuracy: 0.3659
8/8 [==============================] - 0s 3ms/step - loss: 1.5248 - accuracy: 0.3008
1/1 [==============================] - 0s 122ms/step - loss: 1.4190 - accuracy: 0.3571
8/8 [==============================] - 0s 3ms/step - loss: 1.5212 - accuracy: 0.2973
1/1 [==============================] - 0s 117ms/step - loss: 1.5482 - accuracy: 0.3659
8/8 [==============================] - 0s 3ms/step - loss: 1.5311 - accuracy: 0.2703
1/1 [==============================] - 0s 139ms/step - loss: 1.3245 - accuracy: 0.4390
8/8 [==============================] - 0s 4ms/step - loss: 1.5049 - accuracy: 0.3054
1/1 [==============================] - 0s 131ms/step - loss: 1.6631 - accuracy: 0.2195
8/8 [==============================] - 0s 3ms/step - loss: 1.5149 - accuracy: 0.3135
1/1 [==============================] - 0s 131ms/step - loss: 1.3788 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5503 - accuracy: 0.2730
1/1 [==============================] - 0s 134ms/step - loss: 1.3437 - accuracy: 0.4146
8/8 [==============================] - 0s 3ms/step - loss: 1.5244 - accuracy: 0.3162
1/1 [==============================] - 0s 132ms/step - loss: 1.4168 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.4965 - accuracy: 0.3297
1/1 [==============================] - 0s 121ms/step - loss: 1.5158 - accuracy: 0.2683
8/8 [==============================] - 0s 3ms/step - loss: 1.4848 - accuracy: 0.3270
1/1 [==============================] - 0s 126ms/step - loss: 1.7796 - accuracy: 0.2927
8/8 [==============================] - 0s 4ms/step - loss: 1.5010 - accuracy: 0.3054
1/1 [==============================] - 0s 119ms/step - loss: 1.4318 - accuracy: 0.3902
8/8 [==============================] - 0s 4ms/step - loss: 1.5249 - accuracy: 0.2764
1/1 [==============================] - 0s 129ms/step - loss: 1.4052 - accuracy: 0.3810
8/8 [==============================] - 0s 4ms/step - loss: 1.5010 - accuracy: 0.2946
1/1 [==============================] - 0s 119ms/step - loss: 1.6147 - accuracy: 0.3171
8/8 [==============================] - 0s 4ms/step - loss: 1.4941 - accuracy: 0.3270
1/1 [==============================] - 0s 160ms/step - loss: 1.3050 - accuracy: 0.3659
8/8 [==============================] - 1s 6ms/step - loss: 1.4656 - accuracy: 0.3351
1/1 [==============================] - 0s 213ms/step - loss: 1.7787 - accuracy: 0.2439
8/8 [==============================] - 2s 6ms/step - loss: 1.5225 - accuracy: 0.2919
1/1 [==============================] - 0s 134ms/step - loss: 1.3224 - accuracy: 0.3171
8/8 [==============================] - 0s 4ms/step - loss: 1.4963 - accuracy: 0.2892
1/1 [==============================] - 0s 127ms/step - loss: 1.3086 - accuracy: 0.4634
8/8 [==============================] - 0s 4ms/step - loss: 1.4762 - accuracy: 0.3378
1/1 [==============================] - 0s 140ms/step - loss: 1.3352 - accuracy: 0.4146
8/8 [==============================] - 0s 4ms/step - loss: 1.4954 - accuracy: 0.3378
1/1 [==============================] - 0s 132ms/step - loss: 1.5375 - accuracy: 0.2927
8/8 [==============================] - 0s 4ms/step - loss: 1.4837 - accuracy: 0.3054
1/1 [==============================] - 0s 130ms/step - loss: 1.8460 - accuracy: 0.2439
8/8 [==============================] - 0s 4ms/step - loss: 1.4863 - accuracy: 0.3054
1/1 [==============================] - 0s 125ms/step - loss: 1.4203 - accuracy: 0.3659
8/8 [==============================] - 0s 2ms/step - loss: 1.5403 - accuracy: 0.3333
1/1 [==============================] - 0s 120ms/step - loss: 1.5005 - accuracy: 0.3571
8/8 [==============================] - 0s 2ms/step - loss: 1.6259 - accuracy: 0.2054
1/1 [==============================] - 0s 124ms/step - loss: 1.6134 - accuracy: 0.2683
8/8 [==============================] - 0s 3ms/step - loss: 1.5842 - accuracy: 0.3162
1/1 [==============================] - 0s 122ms/step - loss: 1.5234 - accuracy: 0.3902
8/8 [==============================] - 0s 2ms/step - loss: 1.6096 - accuracy: 0.2378
1/1 [==============================] - 0s 120ms/step - loss: 1.5767 - accuracy: 0.2683
8/8 [==============================] - 0s 3ms/step - loss: 1.5678 - accuracy: 0.3054
1/1 [==============================] - 0s 123ms/step - loss: 1.5396 - accuracy: 0.2439
8/8 [==============================] - 0s 3ms/step - loss: 1.5989 - accuracy: 0.2568
1/1 [==============================] - 0s 126ms/step - loss: 1.5539 - accuracy: 0.2195
8/8 [==============================] - 0s 3ms/step - loss: 1.6057 - accuracy: 0.2432
1/1 [==============================] - 0s 122ms/step - loss: 1.5603 - accuracy: 0.2683
8/8 [==============================] - 0s 2ms/step - loss: 1.6180 - accuracy: 0.2216
1/1 [==============================] - 0s 127ms/step - loss: 1.5839 - accuracy: 0.3171
8/8 [==============================] - 0s 2ms/step - loss: 1.5280 - accuracy: 0.2730
1/1 [==============================] - 0s 135ms/step - loss: 1.6846 - accuracy: 0.2439
8/8 [==============================] - 1s 4ms/step - loss: 1.6017 - accuracy: 0.2297
1/1 [==============================] - 0s 196ms/step - loss: 1.5782 - accuracy: 0.2195
8/8 [==============================] - 1s 3ms/step - loss: 1.5631 - accuracy: 0.2575
1/1 [==============================] - 0s 194ms/step - loss: 1.4569 - accuracy: 0.3571
8/8 [==============================] - 1s 3ms/step - loss: 1.5905 - accuracy: 0.2541
1/1 [==============================] - 0s 206ms/step - loss: 1.4459 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.6543 - accuracy: 0.1973
1/1 [==============================] - 0s 122ms/step - loss: 1.4493 - accuracy: 0.3659
8/8 [==============================] - 0s 3ms/step - loss: 1.5933 - accuracy: 0.2541
1/1 [==============================] - 0s 119ms/step - loss: 1.6193 - accuracy: 0.1951
8/8 [==============================] - 0s 3ms/step - loss: 1.5461 - accuracy: 0.2865
1/1 [==============================] - 0s 132ms/step - loss: 1.4013 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5627 - accuracy: 0.2622
1/1 [==============================] - 0s 123ms/step - loss: 1.4224 - accuracy: 0.3659
8/8 [==============================] - 0s 3ms/step - loss: 1.5320 - accuracy: 0.2811
1/1 [==============================] - 0s 117ms/step - loss: 1.4159 - accuracy: 0.4146
8/8 [==============================] - 0s 3ms/step - loss: 1.5462 - accuracy: 0.2676
1/1 [==============================] - 0s 128ms/step - loss: 1.5089 - accuracy: 0.4146
8/8 [==============================] - 0s 3ms/step - loss: 1.5305 - accuracy: 0.2892
1/1 [==============================] - 0s 141ms/step - loss: 1.6746 - accuracy: 0.1951
8/8 [==============================] - 0s 3ms/step - loss: 1.5244 - accuracy: 0.2865
1/1 [==============================] - 0s 129ms/step - loss: 1.4427 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5760 - accuracy: 0.2737
1/1 [==============================] - 0s 120ms/step - loss: 1.4521 - accuracy: 0.3571
8/8 [==============================] - 0s 3ms/step - loss: 1.5059 - accuracy: 0.3081
1/1 [==============================] - 0s 136ms/step - loss: 1.5220 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5275 - accuracy: 0.3000
1/1 [==============================] - 0s 120ms/step - loss: 1.3338 - accuracy: 0.3902
8/8 [==============================] - 0s 3ms/step - loss: 1.5018 - accuracy: 0.3162
1/1 [==============================] - 0s 122ms/step - loss: 1.7264 - accuracy: 0.2195
8/8 [==============================] - 0s 3ms/step - loss: 1.5010 - accuracy: 0.3162
1/1 [==============================] - 0s 132ms/step - loss: 1.3224 - accuracy: 0.2683
8/8 [==============================] - 0s 4ms/step - loss: 1.5208 - accuracy: 0.3108
1/1 [==============================] - 0s 126ms/step - loss: 1.3380 - accuracy: 0.4390
8/8 [==============================] - 2s 5ms/step - loss: 1.5345 - accuracy: 0.3135
1/1 [==============================] - 0s 181ms/step - loss: 1.3838 - accuracy: 0.3171
8/8 [==============================] - 1s 6ms/step - loss: 1.4773 - accuracy: 0.3351
1/1 [==============================] - 0s 200ms/step - loss: 1.5583 - accuracy: 0.2683
8/8 [==============================] - 0s 3ms/step - loss: 1.4760 - accuracy: 0.3189
1/1 [==============================] - 0s 124ms/step - loss: 1.7310 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.5036 - accuracy: 0.3027
1/1 [==============================] - 0s 133ms/step - loss: 1.4300 - accuracy: 0.3415
8/8 [==============================] - 0s 4ms/step - loss: 1.5228 - accuracy: 0.2927
1/1 [==============================] - 0s 125ms/step - loss: 1.4388 - accuracy: 0.3571
8/8 [==============================] - 0s 4ms/step - loss: 1.4596 - accuracy: 0.3405
1/1 [==============================] - 0s 120ms/step - loss: 1.5302 - accuracy: 0.3415
8/8 [==============================] - 0s 4ms/step - loss: 1.5003 - accuracy: 0.2811
1/1 [==============================] - 0s 121ms/step - loss: 1.3398 - accuracy: 0.3902
8/8 [==============================] - 0s 4ms/step - loss: 1.4844 - accuracy: 0.3324
1/1 [==============================] - 0s 126ms/step - loss: 1.6414 - accuracy: 0.2439
8/8 [==============================] - 0s 4ms/step - loss: 1.5039 - accuracy: 0.3162
1/1 [==============================] - 0s 142ms/step - loss: 1.2718 - accuracy: 0.3415
8/8 [==============================] - 0s 4ms/step - loss: 1.4937 - accuracy: 0.3297
1/1 [==============================] - 0s 121ms/step - loss: 1.3282 - accuracy: 0.4878
8/8 [==============================] - 0s 4ms/step - loss: 1.4777 - accuracy: 0.3270
1/1 [==============================] - 0s 128ms/step - loss: 1.3212 - accuracy: 0.3171
8/8 [==============================] - 0s 4ms/step - loss: 1.4668 - accuracy: 0.3270
1/1 [==============================] - 0s 143ms/step - loss: 1.5587 - accuracy: 0.2927
8/8 [==============================] - 0s 4ms/step - loss: 1.4657 - accuracy: 0.3405
1/1 [==============================] - 0s 122ms/step - loss: 1.7965 - accuracy: 0.3415
8/8 [==============================] - 0s 4ms/step - loss: 1.4816 - accuracy: 0.3378
1/1 [==============================] - 0s 133ms/step - loss: 1.4334 - accuracy: 0.3659
9/9 [==============================] - 0s 4ms/step - loss: 1.5015 - accuracy: 0.3285
best parameters for ANN:  {'batch_size': 50, 'nb_epoch': 50, 'unit': 300}
best score for ANN: 0.3479094088077545

best parameters for ANN: {'batch_size': 50, 'nb_epoch': 50, 'unit': 300}

best score for ANN: 0.3479094088077545

Below ANN function with tuned parameters-

def NN_Model_Tuned_CVFull(X_train, X_test, y_train, y_test):
  in_dim = X_train.shape[1]

  tf.random.set_seed(7)

  model = Sequential()
  model.add(Dense(100, activation='relu', input_dim = in_dim))
  model.add(Dropout(0.2))
  model.add(Dense(100, activation='relu'))
  model.add(Dropout(0.2))
  model.add(Dense(100, activation='relu'))
  model.add(Dense(5 , activation='softmax'))

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=0, patience=3)

  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

  print(model.summary())

  history=model.fit(X_train, y_train_cat, validation_split=0.2, epochs = 50, batch_size = 50, callbacks=[early_stopping])

  train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  result_kfold_df= pd.DataFrame({'model': ['Neural Network'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })



  hist= pd.DataFrame(history.history)

  for col in hist.columns:
    print(col)
    plt.plot(hist[col])
    plt.plot(hist[col])
    plt.title('model-'+col)
    plt.ylabel(col)
    plt.xlabel('epoch')
    plt.show()

  return result_kfold_df

NN_Model_Tuned_CVFull(X_train_cvfull, X_test_cvfull, y_train_cvfull, y_test_cvfull)

Model: "sequential_245"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_739 (Dense)           (None, 100)               22000     
                                                                 
 dropout_4 (Dropout)         (None, 100)               0         
                                                                 
 dense_740 (Dense)           (None, 100)               10100     
                                                                 
 dropout_5 (Dropout)         (None, 100)               0         
                                                                 
 dense_741 (Dense)           (None, 100)               10100     
                                                                 
 dense_742 (Dense)           (None, 5)                 505       
                                                                 
=================================================================
Total params: 42,705
Trainable params: 42,705
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
6/6 [==============================] - 2s 83ms/step - loss: 1.5937 - accuracy: 0.2290 - val_loss: 1.5562 - val_accuracy: 0.2576
Epoch 2/50
6/6 [==============================] - 0s 25ms/step - loss: 1.4765 - accuracy: 0.3664 - val_loss: 1.5312 - val_accuracy: 0.3182
Epoch 3/50
6/6 [==============================] - 0s 19ms/step - loss: 1.4233 - accuracy: 0.3359 - val_loss: 1.5272 - val_accuracy: 0.2879
Epoch 4/50
6/6 [==============================] - 0s 20ms/step - loss: 1.3667 - accuracy: 0.4122 - val_loss: 1.5358 - val_accuracy: 0.2121
Epoch 5/50
6/6 [==============================] - 0s 16ms/step - loss: 1.3322 - accuracy: 0.4542 - val_loss: 1.5273 - val_accuracy: 0.2576
Epoch 6/50
6/6 [==============================] - 0s 12ms/step - loss: 1.2947 - accuracy: 0.4237 - val_loss: 1.5065 - val_accuracy: 0.2727
Epoch 7/50
6/6 [==============================] - 0s 11ms/step - loss: 1.2235 - accuracy: 0.4885 - val_loss: 1.4856 - val_accuracy: 0.3030
Epoch 8/50
6/6 [==============================] - 0s 12ms/step - loss: 1.1951 - accuracy: 0.5229 - val_loss: 1.4652 - val_accuracy: 0.3030
Epoch 9/50
6/6 [==============================] - 0s 10ms/step - loss: 1.1224 - accuracy: 0.5687 - val_loss: 1.4378 - val_accuracy: 0.3939
Epoch 10/50
6/6 [==============================] - 0s 12ms/step - loss: 1.0272 - accuracy: 0.5878 - val_loss: 1.4215 - val_accuracy: 0.3636
Epoch 11/50
6/6 [==============================] - 0s 14ms/step - loss: 0.9416 - accuracy: 0.6603 - val_loss: 1.3959 - val_accuracy: 0.3636
Epoch 12/50
6/6 [==============================] - 0s 23ms/step - loss: 0.8735 - accuracy: 0.6718 - val_loss: 1.3780 - val_accuracy: 0.3939
Epoch 13/50
6/6 [==============================] - 0s 19ms/step - loss: 0.7653 - accuracy: 0.7366 - val_loss: 1.3909 - val_accuracy: 0.3636
Epoch 14/50
6/6 [==============================] - 0s 21ms/step - loss: 0.7108 - accuracy: 0.7366 - val_loss: 1.4121 - val_accuracy: 0.3788
Epoch 15/50
6/6 [==============================] - 0s 17ms/step - loss: 0.5961 - accuracy: 0.8092 - val_loss: 1.4568 - val_accuracy: 0.3788
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 5ms/step
11/11 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 5ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.801829	0.457831	0.802949	0.425004

NN_Model_Tuned_CVFull(X_train_cvfull_smote, X_test_cvfull, y_train_cvfull_smote, y_test_cvfull)

Model: "sequential_246"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_743 (Dense)           (None, 100)               22000     
                                                                 
 dropout_6 (Dropout)         (None, 100)               0         
                                                                 
 dense_744 (Dense)           (None, 100)               10100     
                                                                 
 dropout_7 (Dropout)         (None, 100)               0         
                                                                 
 dense_745 (Dense)           (None, 100)               10100     
                                                                 
 dense_746 (Dense)           (None, 5)                 505       
                                                                 
=================================================================
Total params: 42,705
Trainable params: 42,705
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
9/9 [==============================] - 1s 45ms/step - loss: 1.5671 - accuracy: 0.2864 - val_loss: 1.8011 - val_accuracy: 0.0000e+00
Epoch 2/50
9/9 [==============================] - 0s 9ms/step - loss: 1.4665 - accuracy: 0.4000 - val_loss: 2.0131 - val_accuracy: 0.0000e+00
Epoch 3/50
9/9 [==============================] - 0s 12ms/step - loss: 1.3472 - accuracy: 0.4545 - val_loss: 2.0941 - val_accuracy: 0.0000e+00
Epoch 4/50
9/9 [==============================] - 0s 15ms/step - loss: 1.2413 - accuracy: 0.5318 - val_loss: 2.0803 - val_accuracy: 0.0091
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 2ms/step
18/18 [==============================] - 0s 3ms/step
3/3 [==============================] - 0s 3ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.490909	0.409639	0.403699	0.358854

Observation-

i. Smote datasets are better fit. Test accuracy is best around 40%. Best test F1 score is around 35% in CountVectorizer technique.

TFIDF dataset-

Tuned_ANN(X_tfidf_df, y_tfidf_df)

4/4 [==============================] - 2s 5ms/step - loss: 1.6048 - accuracy: 0.2629
1/1 [==============================] - 0s 225ms/step - loss: 1.5932 - accuracy: 0.3571
4/4 [==============================] - 1s 6ms/step - loss: 1.5981 - accuracy: 0.2243
1/1 [==============================] - 0s 203ms/step - loss: 1.5926 - accuracy: 0.2683
4/4 [==============================] - 1s 5ms/step - loss: 1.5963 - accuracy: 0.2622
1/1 [==============================] - 0s 222ms/step - loss: 1.5732 - accuracy: 0.4146
4/4 [==============================] - 0s 3ms/step - loss: 1.5866 - accuracy: 0.3054
1/1 [==============================] - 0s 144ms/step - loss: 1.5994 - accuracy: 0.2195
4/4 [==============================] - 0s 3ms/step - loss: 1.5990 - accuracy: 0.2622
1/1 [==============================] - 0s 131ms/step - loss: 1.5888 - accuracy: 0.3902
4/4 [==============================] - 0s 3ms/step - loss: 1.6064 - accuracy: 0.2189
1/1 [==============================] - 0s 123ms/step - loss: 1.5816 - accuracy: 0.2927
4/4 [==============================] - 0s 3ms/step - loss: 1.5993 - accuracy: 0.2811
1/1 [==============================] - 0s 146ms/step - loss: 1.5906 - accuracy: 0.3171
4/4 [==============================] - 0s 3ms/step - loss: 1.5937 - accuracy: 0.2541
1/1 [==============================] - 0s 128ms/step - loss: 1.5942 - accuracy: 0.1951
4/4 [==============================] - 0s 3ms/step - loss: 1.6181 - accuracy: 0.1676
1/1 [==============================] - 0s 127ms/step - loss: 1.6156 - accuracy: 0.1707
4/4 [==============================] - 0s 3ms/step - loss: 1.6094 - accuracy: 0.2622
1/1 [==============================] - 0s 133ms/step - loss: 1.6167 - accuracy: 0.1463
4/4 [==============================] - 0s 3ms/step - loss: 1.6055 - accuracy: 0.1897
1/1 [==============================] - 0s 140ms/step - loss: 1.5913 - accuracy: 0.2381
4/4 [==============================] - 1s 6ms/step - loss: 1.5868 - accuracy: 0.2649
1/1 [==============================] - 0s 184ms/step - loss: 1.5644 - accuracy: 0.3171
4/4 [==============================] - 1s 5ms/step - loss: 1.6067 - accuracy: 0.2405
1/1 [==============================] - 0s 195ms/step - loss: 1.5924 - accuracy: 0.2683
4/4 [==============================] - 1s 6ms/step - loss: 1.6268 - accuracy: 0.1459
1/1 [==============================] - 0s 187ms/step - loss: 1.6153 - accuracy: 0.1220
4/4 [==============================] - 0s 6ms/step - loss: 1.6058 - accuracy: 0.2622
1/1 [==============================] - 0s 137ms/step - loss: 1.5891 - accuracy: 0.3171
4/4 [==============================] - 0s 4ms/step - loss: 1.6152 - accuracy: 0.1676
1/1 [==============================] - 0s 137ms/step - loss: 1.5895 - accuracy: 0.2439
4/4 [==============================] - 0s 4ms/step - loss: 1.6117 - accuracy: 0.2054
1/1 [==============================] - 0s 146ms/step - loss: 1.5934 - accuracy: 0.3659
4/4 [==============================] - 0s 4ms/step - loss: 1.6044 - accuracy: 0.2243
1/1 [==============================] - 0s 144ms/step - loss: 1.6027 - accuracy: 0.2683
4/4 [==============================] - 0s 4ms/step - loss: 1.5831 - accuracy: 0.2865
1/1 [==============================] - 0s 133ms/step - loss: 1.6148 - accuracy: 0.2439
4/4 [==============================] - 0s 4ms/step - loss: 1.5848 - accuracy: 0.2784
1/1 [==============================] - 0s 127ms/step - loss: 1.5911 - accuracy: 0.2683
4/4 [==============================] - 0s 4ms/step - loss: 1.6083 - accuracy: 0.2195
1/1 [==============================] - 0s 133ms/step - loss: 1.5685 - accuracy: 0.3571
4/4 [==============================] - 0s 5ms/step - loss: 1.5954 - accuracy: 0.2514
1/1 [==============================] - 0s 143ms/step - loss: 1.5547 - accuracy: 0.3171
4/4 [==============================] - 0s 4ms/step - loss: 1.6001 - accuracy: 0.2730
1/1 [==============================] - 0s 126ms/step - loss: 1.5505 - accuracy: 0.3902
4/4 [==============================] - 0s 4ms/step - loss: 1.5842 - accuracy: 0.3270
1/1 [==============================] - 0s 126ms/step - loss: 1.5953 - accuracy: 0.2195
4/4 [==============================] - 0s 4ms/step - loss: 1.6000 - accuracy: 0.2622
1/1 [==============================] - 0s 139ms/step - loss: 1.5715 - accuracy: 0.3171
4/4 [==============================] - 0s 4ms/step - loss: 1.6055 - accuracy: 0.1622
1/1 [==============================] - 0s 143ms/step - loss: 1.5675 - accuracy: 0.1463
4/4 [==============================] - 0s 5ms/step - loss: 1.5959 - accuracy: 0.2297
1/1 [==============================] - 0s 129ms/step - loss: 1.5496 - accuracy: 0.4146
4/4 [==============================] - 1s 7ms/step - loss: 1.5921 - accuracy: 0.2784
1/1 [==============================] - 0s 181ms/step - loss: 1.5708 - accuracy: 0.2927
4/4 [==============================] - 1s 5ms/step - loss: 1.6037 - accuracy: 0.2081
1/1 [==============================] - 0s 176ms/step - loss: 1.5960 - accuracy: 0.3171
4/4 [==============================] - 1s 6ms/step - loss: 1.5846 - accuracy: 0.2838
1/1 [==============================] - 0s 203ms/step - loss: 1.5631 - accuracy: 0.3171
4/4 [==============================] - 0s 4ms/step - loss: 1.5840 - accuracy: 0.2873
1/1 [==============================] - 0s 138ms/step - loss: 1.5343 - accuracy: 0.3333
4/4 [==============================] - 0s 5ms/step - loss: 1.5733 - accuracy: 0.3459
1/1 [==============================] - 0s 131ms/step - loss: 1.5332 - accuracy: 0.3659
4/4 [==============================] - 0s 5ms/step - loss: 1.5942 - accuracy: 0.2838
1/1 [==============================] - 0s 154ms/step - loss: 1.5339 - accuracy: 0.3902
4/4 [==============================] - 0s 5ms/step - loss: 1.5729 - accuracy: 0.3189
1/1 [==============================] - 0s 130ms/step - loss: 1.6026 - accuracy: 0.2195
4/4 [==============================] - 0s 5ms/step - loss: 1.5877 - accuracy: 0.2973
1/1 [==============================] - 0s 128ms/step - loss: 1.5164 - accuracy: 0.3415
4/4 [==============================] - 0s 5ms/step - loss: 1.5930 - accuracy: 0.2568
1/1 [==============================] - 0s 135ms/step - loss: 1.5329 - accuracy: 0.4146
4/4 [==============================] - 0s 5ms/step - loss: 1.5962 - accuracy: 0.2459
1/1 [==============================] - 0s 145ms/step - loss: 1.5455 - accuracy: 0.3171
4/4 [==============================] - 0s 5ms/step - loss: 1.5806 - accuracy: 0.2486
1/1 [==============================] - 0s 132ms/step - loss: 1.5597 - accuracy: 0.3171
4/4 [==============================] - 0s 5ms/step - loss: 1.5785 - accuracy: 0.2730
1/1 [==============================] - 0s 133ms/step - loss: 1.6150 - accuracy: 0.2439
4/4 [==============================] - 0s 5ms/step - loss: 1.5884 - accuracy: 0.2622
1/1 [==============================] - 0s 148ms/step - loss: 1.5349 - accuracy: 0.3659
4/4 [==============================] - 0s 4ms/step - loss: 1.6067 - accuracy: 0.2087
1/1 [==============================] - 0s 145ms/step - loss: 1.6081 - accuracy: 0.1905
4/4 [==============================] - 0s 3ms/step - loss: 1.6325 - accuracy: 0.1541
1/1 [==============================] - 0s 136ms/step - loss: 1.6151 - accuracy: 0.1707
4/4 [==============================] - 0s 3ms/step - loss: 1.5913 - accuracy: 0.3027
1/1 [==============================] - 0s 129ms/step - loss: 1.5612 - accuracy: 0.3902
4/4 [==============================] - 1s 4ms/step - loss: 1.5997 - accuracy: 0.2757
1/1 [==============================] - 0s 216ms/step - loss: 1.6066 - accuracy: 0.1951
4/4 [==============================] - 1s 7ms/step - loss: 1.6027 - accuracy: 0.2405
1/1 [==============================] - 0s 220ms/step - loss: 1.5869 - accuracy: 0.3171
4/4 [==============================] - 1s 3ms/step - loss: 1.6171 - accuracy: 0.2189
1/1 [==============================] - 0s 143ms/step - loss: 1.6013 - accuracy: 0.2683
4/4 [==============================] - 0s 4ms/step - loss: 1.5983 - accuracy: 0.2324
1/1 [==============================] - 0s 139ms/step - loss: 1.5766 - accuracy: 0.2927
4/4 [==============================] - 0s 3ms/step - loss: 1.6029 - accuracy: 0.2297
1/1 [==============================] - 0s 130ms/step - loss: 1.5876 - accuracy: 0.3415
4/4 [==============================] - 0s 4ms/step - loss: 1.6177 - accuracy: 0.1405
1/1 [==============================] - 0s 142ms/step - loss: 1.6101 - accuracy: 0.2439
4/4 [==============================] - 0s 3ms/step - loss: 1.6212 - accuracy: 0.1405
1/1 [==============================] - 0s 127ms/step - loss: 1.6146 - accuracy: 0.1707
4/4 [==============================] - 0s 4ms/step - loss: 1.6082 - accuracy: 0.2087
1/1 [==============================] - 0s 132ms/step - loss: 1.5911 - accuracy: 0.3095
4/4 [==============================] - 0s 5ms/step - loss: 1.6025 - accuracy: 0.2622
1/1 [==============================] - 0s 134ms/step - loss: 1.5826 - accuracy: 0.3415
4/4 [==============================] - 0s 4ms/step - loss: 1.5921 - accuracy: 0.3243
1/1 [==============================] - 0s 148ms/step - loss: 1.5636 - accuracy: 0.4146
4/4 [==============================] - 0s 4ms/step - loss: 1.5982 - accuracy: 0.2405
1/1 [==============================] - 0s 146ms/step - loss: 1.6054 - accuracy: 0.2439
4/4 [==============================] - 0s 4ms/step - loss: 1.5987 - accuracy: 0.2405
1/1 [==============================] - 0s 125ms/step - loss: 1.5501 - accuracy: 0.4634
4/4 [==============================] - 0s 4ms/step - loss: 1.5961 - accuracy: 0.3216
1/1 [==============================] - 0s 136ms/step - loss: 1.5675 - accuracy: 0.4390
4/4 [==============================] - 0s 4ms/step - loss: 1.6113 - accuracy: 0.2135
1/1 [==============================] - 0s 138ms/step - loss: 1.5844 - accuracy: 0.2439
4/4 [==============================] - 0s 6ms/step - loss: 1.6099 - accuracy: 0.1973
1/1 [==============================] - 0s 186ms/step - loss: 1.6003 - accuracy: 0.2195
4/4 [==============================] - 1s 6ms/step - loss: 1.6170 - accuracy: 0.1649
1/1 [==============================] - 0s 194ms/step - loss: 1.6059 - accuracy: 0.1951
4/4 [==============================] - 1s 7ms/step - loss: 1.5891 - accuracy: 0.3189
1/1 [==============================] - 0s 382ms/step - loss: 1.5670 - accuracy: 0.3659
4/4 [==============================] - 1s 8ms/step - loss: 1.5901 - accuracy: 0.2981
1/1 [==============================] - 1s 529ms/step - loss: 1.5554 - accuracy: 0.3810
4/4 [==============================] - 1s 8ms/step - loss: 1.5998 - accuracy: 0.2108
1/1 [==============================] - 0s 203ms/step - loss: 1.5575 - accuracy: 0.3659
4/4 [==============================] - 1s 7ms/step - loss: 1.5830 - accuracy: 0.2784
1/1 [==============================] - 0s 251ms/step - loss: 1.5298 - accuracy: 0.3415
4/4 [==============================] - 1s 7ms/step - loss: 1.5877 - accuracy: 0.3162
1/1 [==============================] - 0s 227ms/step - loss: 1.5979 - accuracy: 0.2195
4/4 [==============================] - 1s 7ms/step - loss: 1.5930 - accuracy: 0.3054
1/1 [==============================] - 0s 217ms/step - loss: 1.5488 - accuracy: 0.3902
4/4 [==============================] - 1s 8ms/step - loss: 1.5938 - accuracy: 0.2459
1/1 [==============================] - 0s 225ms/step - loss: 1.5511 - accuracy: 0.3902
4/4 [==============================] - 1s 7ms/step - loss: 1.5782 - accuracy: 0.2757
1/1 [==============================] - 0s 197ms/step - loss: 1.5249 - accuracy: 0.3415
4/4 [==============================] - 1s 7ms/step - loss: 1.5973 - accuracy: 0.2703
1/1 [==============================] - 0s 154ms/step - loss: 1.5787 - accuracy: 0.3659
4/4 [==============================] - 0s 4ms/step - loss: 1.5780 - accuracy: 0.3324
1/1 [==============================] - 0s 172ms/step - loss: 1.6159 - accuracy: 0.1951
4/4 [==============================] - 1s 6ms/step - loss: 1.5918 - accuracy: 0.2811
1/1 [==============================] - 0s 173ms/step - loss: 1.5544 - accuracy: 0.4146
4/4 [==============================] - 1s 8ms/step - loss: 1.5708 - accuracy: 0.3252
1/1 [==============================] - 0s 198ms/step - loss: 1.5176 - accuracy: 0.3571
4/4 [==============================] - 1s 8ms/step - loss: 1.5903 - accuracy: 0.2784
1/1 [==============================] - 0s 142ms/step - loss: 1.5614 - accuracy: 0.3171
4/4 [==============================] - 0s 5ms/step - loss: 1.5884 - accuracy: 0.3189
1/1 [==============================] - 0s 131ms/step - loss: 1.5230 - accuracy: 0.3902
4/4 [==============================] - 0s 5ms/step - loss: 1.5822 - accuracy: 0.2892
1/1 [==============================] - 0s 129ms/step - loss: 1.6068 - accuracy: 0.2195
4/4 [==============================] - 0s 5ms/step - loss: 1.5793 - accuracy: 0.3189
1/1 [==============================] - 0s 121ms/step - loss: 1.5121 - accuracy: 0.3415
4/4 [==============================] - 0s 5ms/step - loss: 1.5809 - accuracy: 0.2811
1/1 [==============================] - 0s 150ms/step - loss: 1.4995 - accuracy: 0.4634
4/4 [==============================] - 1s 5ms/step - loss: 1.5855 - accuracy: 0.2946
1/1 [==============================] - 0s 136ms/step - loss: 1.5230 - accuracy: 0.3171
4/4 [==============================] - 0s 5ms/step - loss: 1.5866 - accuracy: 0.2703
1/1 [==============================] - 0s 123ms/step - loss: 1.5826 - accuracy: 0.2683
4/4 [==============================] - 0s 5ms/step - loss: 1.5737 - accuracy: 0.3027
1/1 [==============================] - 0s 138ms/step - loss: 1.6075 - accuracy: 0.2683
4/4 [==============================] - 0s 5ms/step - loss: 1.5864 - accuracy: 0.2946
1/1 [==============================] - 0s 131ms/step - loss: 1.5408 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.5971 - accuracy: 0.1978
3/3 [==============================] - 0s 3ms/step - loss: 1.5630 - accuracy: 0.3810
19/19 [==============================] - 0s 2ms/step - loss: 1.6034 - accuracy: 0.2541
3/3 [==============================] - 0s 3ms/step - loss: 1.5753 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.5927 - accuracy: 0.3297
3/3 [==============================] - 0s 3ms/step - loss: 1.5816 - accuracy: 0.2683
19/19 [==============================] - 0s 2ms/step - loss: 1.5932 - accuracy: 0.2622
3/3 [==============================] - 0s 3ms/step - loss: 1.5910 - accuracy: 0.2195
19/19 [==============================] - 1s 3ms/step - loss: 1.5888 - accuracy: 0.2486
3/3 [==============================] - 0s 3ms/step - loss: 1.5370 - accuracy: 0.3659
19/19 [==============================] - 1s 3ms/step - loss: 1.6031 - accuracy: 0.2459
3/3 [==============================] - 0s 5ms/step - loss: 1.5777 - accuracy: 0.1951
19/19 [==============================] - 1s 3ms/step - loss: 1.6192 - accuracy: 0.1973
3/3 [==============================] - 0s 6ms/step - loss: 1.6035 - accuracy: 0.3415
19/19 [==============================] - 1s 2ms/step - loss: 1.5864 - accuracy: 0.3243
3/3 [==============================] - 0s 5ms/step - loss: 1.5746 - accuracy: 0.2927
19/19 [==============================] - 1s 4ms/step - loss: 1.5676 - accuracy: 0.3378
3/3 [==============================] - 0s 3ms/step - loss: 1.6031 - accuracy: 0.2195
19/19 [==============================] - 1s 3ms/step - loss: 1.6011 - accuracy: 0.2838
3/3 [==============================] - 0s 7ms/step - loss: 1.5809 - accuracy: 0.3415
19/19 [==============================] - 1s 4ms/step - loss: 1.5921 - accuracy: 0.2412
3/3 [==============================] - 0s 3ms/step - loss: 1.5553 - accuracy: 0.3095
19/19 [==============================] - 1s 2ms/step - loss: 1.5695 - accuracy: 0.2973
3/3 [==============================] - 0s 6ms/step - loss: 1.5382 - accuracy: 0.3415
19/19 [==============================] - 1s 4ms/step - loss: 1.5869 - accuracy: 0.3027
3/3 [==============================] - 0s 3ms/step - loss: 1.5335 - accuracy: 0.3902
19/19 [==============================] - 1s 2ms/step - loss: 1.5477 - accuracy: 0.3324
3/3 [==============================] - 0s 6ms/step - loss: 1.6156 - accuracy: 0.2195
19/19 [==============================] - 1s 4ms/step - loss: 1.5835 - accuracy: 0.2892
3/3 [==============================] - 0s 7ms/step - loss: 1.5191 - accuracy: 0.4146
19/19 [==============================] - 3s 8ms/step - loss: 1.5847 - accuracy: 0.2865
3/3 [==============================] - 1s 7ms/step - loss: 1.5101 - accuracy: 0.4390
19/19 [==============================] - 1s 4ms/step - loss: 1.5800 - accuracy: 0.2946
3/3 [==============================] - 0s 6ms/step - loss: 1.5295 - accuracy: 0.3415
19/19 [==============================] - 1s 4ms/step - loss: 1.5827 - accuracy: 0.2351
3/3 [==============================] - 0s 4ms/step - loss: 1.5695 - accuracy: 0.1951
19/19 [==============================] - 1s 5ms/step - loss: 1.5708 - accuracy: 0.2568
3/3 [==============================] - 0s 7ms/step - loss: 1.6179 - accuracy: 0.2683
19/19 [==============================] - 1s 4ms/step - loss: 1.5660 - accuracy: 0.3270
3/3 [==============================] - 0s 4ms/step - loss: 1.5188 - accuracy: 0.3902
19/19 [==============================] - 1s 4ms/step - loss: 1.5339 - accuracy: 0.2846
3/3 [==============================] - 0s 6ms/step - loss: 1.4577 - accuracy: 0.3571
19/19 [==============================] - 1s 4ms/step - loss: 1.5349 - accuracy: 0.3243
3/3 [==============================] - 0s 6ms/step - loss: 1.4826 - accuracy: 0.3415
19/19 [==============================] - 1s 2ms/step - loss: 1.5441 - accuracy: 0.3432
3/3 [==============================] - 0s 3ms/step - loss: 1.4236 - accuracy: 0.3902
19/19 [==============================] - 0s 2ms/step - loss: 1.5406 - accuracy: 0.3135
3/3 [==============================] - 0s 3ms/step - loss: 1.6482 - accuracy: 0.2195
19/19 [==============================] - 1s 3ms/step - loss: 1.5489 - accuracy: 0.3135
3/3 [==============================] - 0s 3ms/step - loss: 1.4222 - accuracy: 0.3415
19/19 [==============================] - 1s 3ms/step - loss: 1.5628 - accuracy: 0.3000
3/3 [==============================] - 0s 3ms/step - loss: 1.4463 - accuracy: 0.4390
19/19 [==============================] - 1s 4ms/step - loss: 1.5457 - accuracy: 0.3108
3/3 [==============================] - 0s 3ms/step - loss: 1.4443 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.5491 - accuracy: 0.2838
3/3 [==============================] - 0s 2ms/step - loss: 1.5193 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5338 - accuracy: 0.3000
3/3 [==============================] - 1s 3ms/step - loss: 1.6786 - accuracy: 0.2927
19/19 [==============================] - 1s 5ms/step - loss: 1.5410 - accuracy: 0.3351
3/3 [==============================] - 0s 4ms/step - loss: 1.5017 - accuracy: 0.3659
19/19 [==============================] - 0s 3ms/step - loss: 1.5402 - accuracy: 0.2764
3/3 [==============================] - 0s 4ms/step - loss: 1.4527 - accuracy: 0.3571
19/19 [==============================] - 0s 3ms/step - loss: 1.5208 - accuracy: 0.3216
3/3 [==============================] - 0s 3ms/step - loss: 1.4289 - accuracy: 0.3415
19/19 [==============================] - 1s 5ms/step - loss: 1.5358 - accuracy: 0.3216
3/3 [==============================] - 0s 4ms/step - loss: 1.3856 - accuracy: 0.3902
19/19 [==============================] - 1s 4ms/step - loss: 1.5086 - accuracy: 0.3351
3/3 [==============================] - 0s 4ms/step - loss: 1.7205 - accuracy: 0.2195
19/19 [==============================] - 0s 3ms/step - loss: 1.5286 - accuracy: 0.3243
3/3 [==============================] - 0s 3ms/step - loss: 1.3606 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5157 - accuracy: 0.2973
3/3 [==============================] - 0s 3ms/step - loss: 1.3711 - accuracy: 0.4390
19/19 [==============================] - 0s 3ms/step - loss: 1.5161 - accuracy: 0.3324
3/3 [==============================] - 0s 4ms/step - loss: 1.3955 - accuracy: 0.3171
19/19 [==============================] - 0s 3ms/step - loss: 1.5185 - accuracy: 0.3162
3/3 [==============================] - 0s 5ms/step - loss: 1.5073 - accuracy: 0.2927
19/19 [==============================] - 1s 5ms/step - loss: 1.5078 - accuracy: 0.3486
3/3 [==============================] - 0s 6ms/step - loss: 1.7208 - accuracy: 0.2927
19/19 [==============================] - 1s 5ms/step - loss: 1.5233 - accuracy: 0.3243
3/3 [==============================] - 0s 3ms/step - loss: 1.4472 - accuracy: 0.3659
19/19 [==============================] - 1s 2ms/step - loss: 1.5889 - accuracy: 0.2927
3/3 [==============================] - 0s 3ms/step - loss: 1.5600 - accuracy: 0.3333
19/19 [==============================] - 0s 2ms/step - loss: 1.5850 - accuracy: 0.3243
3/3 [==============================] - 0s 4ms/step - loss: 1.5639 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.5990 - accuracy: 0.2432
3/3 [==============================] - 0s 3ms/step - loss: 1.5606 - accuracy: 0.3902
19/19 [==============================] - 0s 2ms/step - loss: 1.5761 - accuracy: 0.2568
3/3 [==============================] - 0s 3ms/step - loss: 1.6086 - accuracy: 0.0732
19/19 [==============================] - 0s 2ms/step - loss: 1.5831 - accuracy: 0.3297
3/3 [==============================] - 0s 4ms/step - loss: 1.5359 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5934 - accuracy: 0.3216
3/3 [==============================] - 0s 4ms/step - loss: 1.5529 - accuracy: 0.4390
19/19 [==============================] - 0s 2ms/step - loss: 1.5875 - accuracy: 0.2811
3/3 [==============================] - 0s 3ms/step - loss: 1.5583 - accuracy: 0.3659
19/19 [==============================] - 1s 5ms/step - loss: 1.5713 - accuracy: 0.2514
3/3 [==============================] - 0s 6ms/step - loss: 1.5745 - accuracy: 0.1707
19/19 [==============================] - 1s 4ms/step - loss: 1.5900 - accuracy: 0.2405
3/3 [==============================] - 0s 6ms/step - loss: 1.6259 - accuracy: 0.1707
19/19 [==============================] - 1s 6ms/step - loss: 1.5949 - accuracy: 0.2486
3/3 [==============================] - 0s 4ms/step - loss: 1.5533 - accuracy: 0.4878
19/19 [==============================] - 1s 4ms/step - loss: 1.5881 - accuracy: 0.2304
3/3 [==============================] - 1s 6ms/step - loss: 1.5332 - accuracy: 0.3571
19/19 [==============================] - 2s 8ms/step - loss: 1.5632 - accuracy: 0.2919
3/3 [==============================] - 1s 5ms/step - loss: 1.4825 - accuracy: 0.3415
19/19 [==============================] - 1s 5ms/step - loss: 1.5736 - accuracy: 0.3189
3/3 [==============================] - 0s 6ms/step - loss: 1.4920 - accuracy: 0.3902
19/19 [==============================] - 1s 4ms/step - loss: 1.5640 - accuracy: 0.3405
3/3 [==============================] - 0s 9ms/step - loss: 1.6201 - accuracy: 0.2195
19/19 [==============================] - 1s 4ms/step - loss: 1.5728 - accuracy: 0.3162
3/3 [==============================] - 0s 5ms/step - loss: 1.4978 - accuracy: 0.3171
19/19 [==============================] - 1s 4ms/step - loss: 1.5778 - accuracy: 0.2405
3/3 [==============================] - 0s 5ms/step - loss: 1.4952 - accuracy: 0.4390
19/19 [==============================] - 1s 3ms/step - loss: 1.5662 - accuracy: 0.3432
3/3 [==============================] - 0s 5ms/step - loss: 1.4923 - accuracy: 0.3171
19/19 [==============================] - 1s 4ms/step - loss: 1.5821 - accuracy: 0.2378
3/3 [==============================] - 0s 5ms/step - loss: 1.5667 - accuracy: 0.2927
19/19 [==============================] - 1s 4ms/step - loss: 1.5572 - accuracy: 0.3351
3/3 [==============================] - 0s 4ms/step - loss: 1.6110 - accuracy: 0.2927
19/19 [==============================] - 1s 5ms/step - loss: 1.5602 - accuracy: 0.3108
3/3 [==============================] - 0s 6ms/step - loss: 1.4965 - accuracy: 0.3659
19/19 [==============================] - 1s 7ms/step - loss: 1.5468 - accuracy: 0.3171
3/3 [==============================] - 0s 4ms/step - loss: 1.4608 - accuracy: 0.3571
19/19 [==============================] - 2s 4ms/step - loss: 1.5446 - accuracy: 0.3297
3/3 [==============================] - 0s 4ms/step - loss: 1.4868 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5535 - accuracy: 0.3054
3/3 [==============================] - 0s 3ms/step - loss: 1.4356 - accuracy: 0.3902
19/19 [==============================] - 1s 4ms/step - loss: 1.5330 - accuracy: 0.3459
3/3 [==============================] - 0s 7ms/step - loss: 1.6440 - accuracy: 0.2195
19/19 [==============================] - 1s 5ms/step - loss: 1.5429 - accuracy: 0.3324
3/3 [==============================] - 0s 5ms/step - loss: 1.4089 - accuracy: 0.3415
19/19 [==============================] - 1s 5ms/step - loss: 1.5501 - accuracy: 0.3000
3/3 [==============================] - 0s 7ms/step - loss: 1.4146 - accuracy: 0.4390
19/19 [==============================] - 1s 5ms/step - loss: 1.5484 - accuracy: 0.3108
3/3 [==============================] - 0s 5ms/step - loss: 1.4473 - accuracy: 0.3171
19/19 [==============================] - 1s 5ms/step - loss: 1.5408 - accuracy: 0.3324
3/3 [==============================] - 1s 8ms/step - loss: 1.5362 - accuracy: 0.2683
19/19 [==============================] - 1s 8ms/step - loss: 1.5460 - accuracy: 0.3270
3/3 [==============================] - 1s 7ms/step - loss: 1.6290 - accuracy: 0.2927
19/19 [==============================] - 1s 5ms/step - loss: 1.5522 - accuracy: 0.3081
3/3 [==============================] - 0s 7ms/step - loss: 1.4754 - accuracy: 0.3902
19/19 [==============================] - 1s 6ms/step - loss: 1.5178 - accuracy: 0.3062
3/3 [==============================] - 0s 3ms/step - loss: 1.4372 - accuracy: 0.3571
19/19 [==============================] - 1s 6ms/step - loss: 1.5377 - accuracy: 0.3081
3/3 [==============================] - 0s 6ms/step - loss: 1.4731 - accuracy: 0.3415
19/19 [==============================] - 0s 3ms/step - loss: 1.5141 - accuracy: 0.3270
3/3 [==============================] - 0s 3ms/step - loss: 1.3571 - accuracy: 0.3902
19/19 [==============================] - 1s 5ms/step - loss: 1.5130 - accuracy: 0.3297
3/3 [==============================] - 0s 9ms/step - loss: 1.6861 - accuracy: 0.2195
19/19 [==============================] - 1s 7ms/step - loss: 1.5175 - accuracy: 0.3108
3/3 [==============================] - 0s 4ms/step - loss: 1.3774 - accuracy: 0.3415
19/19 [==============================] - 1s 7ms/step - loss: 1.5378 - accuracy: 0.2892
3/3 [==============================] - 0s 7ms/step - loss: 1.3769 - accuracy: 0.4390
19/19 [==============================] - 1s 6ms/step - loss: 1.5180 - accuracy: 0.3324
3/3 [==============================] - 0s 7ms/step - loss: 1.4024 - accuracy: 0.3171
19/19 [==============================] - 1s 5ms/step - loss: 1.5346 - accuracy: 0.2973
3/3 [==============================] - 0s 9ms/step - loss: 1.5353 - accuracy: 0.2927
19/19 [==============================] - 1s 5ms/step - loss: 1.5052 - accuracy: 0.3108
3/3 [==============================] - 0s 4ms/step - loss: 1.7380 - accuracy: 0.2927
19/19 [==============================] - 1s 4ms/step - loss: 1.5316 - accuracy: 0.3162
3/3 [==============================] - 0s 4ms/step - loss: 1.4343 - accuracy: 0.3659
8/8 [==============================] - 0s 2ms/step - loss: 1.5968 - accuracy: 0.2954
1/1 [==============================] - 0s 136ms/step - loss: 1.5738 - accuracy: 0.3810
8/8 [==============================] - 0s 3ms/step - loss: 1.6022 - accuracy: 0.2946
1/1 [==============================] - 0s 148ms/step - loss: 1.5921 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.5885 - accuracy: 0.3243
1/1 [==============================] - 0s 131ms/step - loss: 1.5609 - accuracy: 0.4146
8/8 [==============================] - 0s 4ms/step - loss: 1.5938 - accuracy: 0.2622
1/1 [==============================] - 0s 135ms/step - loss: 1.6167 - accuracy: 0.1463
8/8 [==============================] - 0s 3ms/step - loss: 1.6068 - accuracy: 0.2324
1/1 [==============================] - 0s 129ms/step - loss: 1.6018 - accuracy: 0.3902
8/8 [==============================] - 0s 3ms/step - loss: 1.6118 - accuracy: 0.1757
1/1 [==============================] - 0s 133ms/step - loss: 1.6007 - accuracy: 0.1707
8/8 [==============================] - 0s 2ms/step - loss: 1.6075 - accuracy: 0.1838
1/1 [==============================] - 0s 141ms/step - loss: 1.5775 - accuracy: 0.3902
8/8 [==============================] - 1s 4ms/step - loss: 1.5953 - accuracy: 0.2973
1/1 [==============================] - 0s 194ms/step - loss: 1.5913 - accuracy: 0.2683
8/8 [==============================] - 1s 4ms/step - loss: 1.5878 - accuracy: 0.2946
1/1 [==============================] - 0s 189ms/step - loss: 1.6071 - accuracy: 0.2195
8/8 [==============================] - 1s 4ms/step - loss: 1.5993 - accuracy: 0.2486
1/1 [==============================] - 0s 156ms/step - loss: 1.5847 - accuracy: 0.2439
8/8 [==============================] - 0s 3ms/step - loss: 1.5794 - accuracy: 0.3198
1/1 [==============================] - 0s 168ms/step - loss: 1.5516 - accuracy: 0.3095
8/8 [==============================] - 0s 3ms/step - loss: 1.5891 - accuracy: 0.3216
1/1 [==============================] - 0s 208ms/step - loss: 1.5665 - accuracy: 0.3415
8/8 [==============================] - 1s 5ms/step - loss: 1.5939 - accuracy: 0.2622
1/1 [==============================] - 0s 243ms/step - loss: 1.5583 - accuracy: 0.3171
8/8 [==============================] - 1s 5ms/step - loss: 1.5916 - accuracy: 0.2865
1/1 [==============================] - 0s 228ms/step - loss: 1.6036 - accuracy: 0.1951
8/8 [==============================] - 2s 4ms/step - loss: 1.5808 - accuracy: 0.3351
1/1 [==============================] - 0s 232ms/step - loss: 1.5409 - accuracy: 0.3415
8/8 [==============================] - 1s 5ms/step - loss: 1.5847 - accuracy: 0.2892
1/1 [==============================] - 0s 204ms/step - loss: 1.5350 - accuracy: 0.3415
8/8 [==============================] - 1s 4ms/step - loss: 1.5841 - accuracy: 0.2973
1/1 [==============================] - 0s 219ms/step - loss: 1.5456 - accuracy: 0.3659
8/8 [==============================] - 1s 3ms/step - loss: 1.6082 - accuracy: 0.1757
1/1 [==============================] - 0s 128ms/step - loss: 1.5940 - accuracy: 0.1707
8/8 [==============================] - 1s 4ms/step - loss: 1.5836 - accuracy: 0.2838
1/1 [==============================] - 0s 173ms/step - loss: 1.6099 - accuracy: 0.1951
8/8 [==============================] - 1s 3ms/step - loss: 1.5928 - accuracy: 0.2324
1/1 [==============================] - 0s 175ms/step - loss: 1.5573 - accuracy: 0.2927
8/8 [==============================] - 1s 6ms/step - loss: 1.5736 - accuracy: 0.2818
1/1 [==============================] - 0s 197ms/step - loss: 1.5276 - accuracy: 0.2381
8/8 [==============================] - 0s 3ms/step - loss: 1.5796 - accuracy: 0.2919
1/1 [==============================] - 0s 130ms/step - loss: 1.5203 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5805 - accuracy: 0.3054
1/1 [==============================] - 0s 203ms/step - loss: 1.4980 - accuracy: 0.4146
8/8 [==============================] - 1s 5ms/step - loss: 1.5646 - accuracy: 0.3324
1/1 [==============================] - 0s 213ms/step - loss: 1.6218 - accuracy: 0.2195
8/8 [==============================] - 1s 6ms/step - loss: 1.5984 - accuracy: 0.2514
1/1 [==============================] - 0s 222ms/step - loss: 1.5393 - accuracy: 0.3415
8/8 [==============================] - 1s 6ms/step - loss: 1.5969 - accuracy: 0.1946
1/1 [==============================] - 0s 224ms/step - loss: 1.5389 - accuracy: 0.2927
8/8 [==============================] - 1s 5ms/step - loss: 1.5764 - accuracy: 0.3189
1/1 [==============================] - 0s 228ms/step - loss: 1.5165 - accuracy: 0.2927
8/8 [==============================] - 1s 6ms/step - loss: 1.5822 - accuracy: 0.2838
1/1 [==============================] - 0s 209ms/step - loss: 1.5708 - accuracy: 0.2927
8/8 [==============================] - 1s 6ms/step - loss: 1.5734 - accuracy: 0.2892
1/1 [==============================] - 0s 202ms/step - loss: 1.6134 - accuracy: 0.2683
8/8 [==============================] - 0s 3ms/step - loss: 1.5775 - accuracy: 0.2703
1/1 [==============================] - 0s 124ms/step - loss: 1.5215 - accuracy: 0.3415
8/8 [==============================] - 0s 6ms/step - loss: 1.5756 - accuracy: 0.2629
1/1 [==============================] - 0s 204ms/step - loss: 1.5127 - accuracy: 0.3333
8/8 [==============================] - 1s 6ms/step - loss: 1.5665 - accuracy: 0.2973
1/1 [==============================] - 0s 196ms/step - loss: 1.5106 - accuracy: 0.3415
8/8 [==============================] - 1s 5ms/step - loss: 1.5639 - accuracy: 0.3108
1/1 [==============================] - 0s 210ms/step - loss: 1.4653 - accuracy: 0.3902
8/8 [==============================] - 1s 3ms/step - loss: 1.5542 - accuracy: 0.3378
1/1 [==============================] - 0s 133ms/step - loss: 1.6238 - accuracy: 0.2195
8/8 [==============================] - 1s 6ms/step - loss: 1.5639 - accuracy: 0.3378
1/1 [==============================] - 0s 227ms/step - loss: 1.4515 - accuracy: 0.3415
8/8 [==============================] - 1s 7ms/step - loss: 1.5653 - accuracy: 0.2297
1/1 [==============================] - 0s 232ms/step - loss: 1.4615 - accuracy: 0.3902
8/8 [==============================] - 1s 6ms/step - loss: 1.5750 - accuracy: 0.2649
1/1 [==============================] - 0s 245ms/step - loss: 1.4958 - accuracy: 0.3171
8/8 [==============================] - 1s 7ms/step - loss: 1.5550 - accuracy: 0.2297
1/1 [==============================] - 0s 240ms/step - loss: 1.5383 - accuracy: 0.2927
8/8 [==============================] - 1s 7ms/step - loss: 1.5635 - accuracy: 0.2595
1/1 [==============================] - 0s 201ms/step - loss: 1.6225 - accuracy: 0.2439
8/8 [==============================] - 1s 7ms/step - loss: 1.5709 - accuracy: 0.3108
1/1 [==============================] - 0s 234ms/step - loss: 1.4982 - accuracy: 0.3659
8/8 [==============================] - 1s 4ms/step - loss: 1.5990 - accuracy: 0.2385
1/1 [==============================] - 0s 226ms/step - loss: 1.5905 - accuracy: 0.2857
8/8 [==============================] - 2s 12ms/step - loss: 1.6005 - accuracy: 0.2324
1/1 [==============================] - 1s 728ms/step - loss: 1.5998 - accuracy: 0.1951
8/8 [==============================] - 2s 6ms/step - loss: 1.5887 - accuracy: 0.2757
1/1 [==============================] - 0s 345ms/step - loss: 1.5411 - accuracy: 0.3659
8/8 [==============================] - 1s 4ms/step - loss: 1.6049 - accuracy: 0.2541
1/1 [==============================] - 0s 279ms/step - loss: 1.6148 - accuracy: 0.1463
8/8 [==============================] - 1s 5ms/step - loss: 1.5936 - accuracy: 0.2081
1/1 [==============================] - 0s 415ms/step - loss: 1.5763 - accuracy: 0.1707
8/8 [==============================] - 1s 5ms/step - loss: 1.5827 - accuracy: 0.2568
1/1 [==============================] - 0s 306ms/step - loss: 1.5533 - accuracy: 0.3415
8/8 [==============================] - 1s 5ms/step - loss: 1.6051 - accuracy: 0.2270
1/1 [==============================] - 0s 409ms/step - loss: 1.5801 - accuracy: 0.3171
8/8 [==============================] - 4s 5ms/step - loss: 1.6053 - accuracy: 0.2541
1/1 [==============================] - 1s 510ms/step - loss: 1.5895 - accuracy: 0.3171
8/8 [==============================] - 1s 3ms/step - loss: 1.5936 - accuracy: 0.2946
1/1 [==============================] - 0s 167ms/step - loss: 1.6083 - accuracy: 0.2195
8/8 [==============================] - 1s 3ms/step - loss: 1.6109 - accuracy: 0.2162
1/1 [==============================] - 0s 221ms/step - loss: 1.5873 - accuracy: 0.3659
8/8 [==============================] - 1s 5ms/step - loss: 1.5879 - accuracy: 0.2412
1/1 [==============================] - 0s 209ms/step - loss: 1.5556 - accuracy: 0.2857
8/8 [==============================] - 1s 6ms/step - loss: 1.6031 - accuracy: 0.2270
1/1 [==============================] - 0s 206ms/step - loss: 1.5774 - accuracy: 0.3171
8/8 [==============================] - 1s 6ms/step - loss: 1.5997 - accuracy: 0.2541
1/1 [==============================] - 0s 228ms/step - loss: 1.5635 - accuracy: 0.3659
8/8 [==============================] - 1s 5ms/step - loss: 1.5793 - accuracy: 0.3108
1/1 [==============================] - 0s 377ms/step - loss: 1.6257 - accuracy: 0.1707
8/8 [==============================] - 1s 4ms/step - loss: 1.5906 - accuracy: 0.3000
1/1 [==============================] - 0s 285ms/step - loss: 1.5399 - accuracy: 0.3902
8/8 [==============================] - 2s 8ms/step - loss: 1.5913 - accuracy: 0.2892
1/1 [==============================] - 0s 404ms/step - loss: 1.5400 - accuracy: 0.5122
8/8 [==============================] - 1s 5ms/step - loss: 1.5867 - accuracy: 0.2946
1/1 [==============================] - 0s 212ms/step - loss: 1.5523 - accuracy: 0.2927
8/8 [==============================] - 1s 5ms/step - loss: 1.6014 - accuracy: 0.1919
1/1 [==============================] - 0s 222ms/step - loss: 1.5856 - accuracy: 0.2683
8/8 [==============================] - 1s 4ms/step - loss: 1.5835 - accuracy: 0.2703
1/1 [==============================] - 0s 226ms/step - loss: 1.6113 - accuracy: 0.2439
8/8 [==============================] - 1s 5ms/step - loss: 1.6102 - accuracy: 0.2108
1/1 [==============================] - 0s 237ms/step - loss: 1.5719 - accuracy: 0.3415
8/8 [==============================] - 1s 8ms/step - loss: 1.5761 - accuracy: 0.2818
1/1 [==============================] - 0s 241ms/step - loss: 1.5187 - accuracy: 0.3571
8/8 [==============================] - 1s 6ms/step - loss: 1.5842 - accuracy: 0.2703
1/1 [==============================] - 0s 239ms/step - loss: 1.5289 - accuracy: 0.3415
8/8 [==============================] - 1s 6ms/step - loss: 1.5885 - accuracy: 0.2892
1/1 [==============================] - 0s 214ms/step - loss: 1.5322 - accuracy: 0.3902
8/8 [==============================] - 1s 6ms/step - loss: 1.5632 - accuracy: 0.3297
1/1 [==============================] - 0s 169ms/step - loss: 1.6081 - accuracy: 0.2195
8/8 [==============================] - 0s 3ms/step - loss: 1.5795 - accuracy: 0.2838
1/1 [==============================] - 0s 132ms/step - loss: 1.5065 - accuracy: 0.3415
8/8 [==============================] - 1s 6ms/step - loss: 1.5811 - accuracy: 0.2297
1/1 [==============================] - 0s 182ms/step - loss: 1.5161 - accuracy: 0.4390
8/8 [==============================] - 1s 5ms/step - loss: 1.5808 - accuracy: 0.2649
1/1 [==============================] - 0s 182ms/step - loss: 1.5039 - accuracy: 0.4146
8/8 [==============================] - 1s 6ms/step - loss: 1.5663 - accuracy: 0.2541
1/1 [==============================] - 0s 208ms/step - loss: 1.5514 - accuracy: 0.2927
8/8 [==============================] - 1s 3ms/step - loss: 1.5839 - accuracy: 0.2378
1/1 [==============================] - 0s 126ms/step - loss: 1.6209 - accuracy: 0.2683
8/8 [==============================] - 0s 4ms/step - loss: 1.5779 - accuracy: 0.3027
1/1 [==============================] - 0s 136ms/step - loss: 1.5370 - accuracy: 0.3415
8/8 [==============================] - 1s 7ms/step - loss: 1.5622 - accuracy: 0.2791
1/1 [==============================] - 0s 221ms/step - loss: 1.4806 - accuracy: 0.3571
8/8 [==============================] - 1s 8ms/step - loss: 1.5678 - accuracy: 0.3081
1/1 [==============================] - 0s 261ms/step - loss: 1.5181 - accuracy: 0.3415
8/8 [==============================] - 1s 6ms/step - loss: 1.5594 - accuracy: 0.3324
1/1 [==============================] - 0s 257ms/step - loss: 1.4524 - accuracy: 0.3902
8/8 [==============================] - 1s 6ms/step - loss: 1.5523 - accuracy: 0.3000
1/1 [==============================] - 0s 233ms/step - loss: 1.6135 - accuracy: 0.2195
8/8 [==============================] - 1s 5ms/step - loss: 1.5737 - accuracy: 0.2919
1/1 [==============================] - 0s 234ms/step - loss: 1.4764 - accuracy: 0.3415
8/8 [==============================] - 1s 7ms/step - loss: 1.5562 - accuracy: 0.3270
1/1 [==============================] - 0s 213ms/step - loss: 1.4506 - accuracy: 0.4878
8/8 [==============================] - 1s 6ms/step - loss: 1.5560 - accuracy: 0.2730
1/1 [==============================] - 0s 206ms/step - loss: 1.4534 - accuracy: 0.3171
8/8 [==============================] - 1s 5ms/step - loss: 1.5531 - accuracy: 0.3459
1/1 [==============================] - 0s 344ms/step - loss: 1.5273 - accuracy: 0.2927
8/8 [==============================] - 1s 8ms/step - loss: 1.5602 - accuracy: 0.3324
1/1 [==============================] - 0s 289ms/step - loss: 1.6255 - accuracy: 0.2927
8/8 [==============================] - 1s 7ms/step - loss: 1.5710 - accuracy: 0.3054
1/1 [==============================] - 1s 1s/step - loss: 1.5036 - accuracy: 0.3659
9/9 [==============================] - 1s 3ms/step - loss: 1.5793 - accuracy: 0.2555
best parameters for ANN:  {'batch_size': 50, 'nb_epoch': 50, 'unit': 200}
best score for ANN: 0.34059233367443087

Best Parameters for TFIDF dataset-

best parameters for ANN: {'batch_size': 50, 'nb_epoch': 50, 'unit': 200}

best score for ANN: 0.34059233367443087

def NN_Model_Tuned_TFIDF(X_train, X_test, y_train, y_test):
  in_dim = X_train.shape[1]

  tf.random.set_seed(7)

  model = Sequential()
  model.add(Dense(100, activation='relu', input_dim = in_dim))
  model.add(Dropout(0.2))
  model.add(Dense(100, activation='relu'))
  model.add(Dense(5 , activation='softmax'))

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=0, patience=3)

  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

  print(model.summary())

  history=model.fit(X_train, y_train_cat, validation_split=0.2, epochs = 50, batch_size = 50, callbacks=[early_stopping])

  train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  result_kfold_df= pd.DataFrame({'model': ['Neural Network'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })



  hist= pd.DataFrame(history.history)

  for col in hist.columns:
    print(col)
    plt.plot(hist[col])
    plt.plot(hist[col])
    plt.title('model-'+col)
    plt.ylabel(col)
    plt.xlabel('epoch')
    plt.show()

  return result_kfold_df

NN_Model_Tuned_TFIDF(X_train_tfidf, X_test_tfidf, y_train_tfidf, y_test_tfidf)

Model: "sequential_488"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_1470 (Dense)          (None, 100)               20100     
                                                                 
 dropout_8 (Dropout)         (None, 100)               0         
                                                                 
 dense_1471 (Dense)          (None, 100)               10100     
                                                                 
 dense_1472 (Dense)          (None, 5)                 505       
                                                                 
=================================================================
Total params: 30,705
Trainable params: 30,705
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
6/6 [==============================] - 1s 53ms/step - loss: 1.6010 - accuracy: 0.2939 - val_loss: 1.5892 - val_accuracy: 0.3182
Epoch 2/50
6/6 [==============================] - 0s 9ms/step - loss: 1.5516 - accuracy: 0.3435 - val_loss: 1.5687 - val_accuracy: 0.3333
Epoch 3/50
6/6 [==============================] - 0s 9ms/step - loss: 1.5158 - accuracy: 0.3511 - val_loss: 1.5503 - val_accuracy: 0.3333
Epoch 4/50
6/6 [==============================] - 0s 11ms/step - loss: 1.4761 - accuracy: 0.3435 - val_loss: 1.5348 - val_accuracy: 0.3333
Epoch 5/50
6/6 [==============================] - 0s 10ms/step - loss: 1.4359 - accuracy: 0.3397 - val_loss: 1.5252 - val_accuracy: 0.3333
Epoch 6/50
6/6 [==============================] - 0s 10ms/step - loss: 1.4003 - accuracy: 0.3397 - val_loss: 1.5252 - val_accuracy: 0.3333
Epoch 7/50
6/6 [==============================] - 0s 10ms/step - loss: 1.3695 - accuracy: 0.3397 - val_loss: 1.5289 - val_accuracy: 0.3333
Epoch 8/50
6/6 [==============================] - 0s 10ms/step - loss: 1.3348 - accuracy: 0.3435 - val_loss: 1.5308 - val_accuracy: 0.3333
Epoch 9/50
6/6 [==============================] - 0s 10ms/step - loss: 1.2993 - accuracy: 0.3588 - val_loss: 1.5287 - val_accuracy: 0.3333
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.375	0.337349	0.246895	0.171742

NN_Model_Tuned_TFIDF(X_train_tfidf_smote, X_test_tfidf, y_train_tfidf_smote, y_test_tfidf)

Model: "sequential_489"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_1473 (Dense)          (None, 100)               20100     
                                                                 
 dropout_9 (Dropout)         (None, 100)               0         
                                                                 
 dense_1474 (Dense)          (None, 100)               10100     
                                                                 
 dense_1475 (Dense)          (None, 5)                 505       
                                                                 
=================================================================
Total params: 30,705
Trainable params: 30,705
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
9/9 [==============================] - 1s 26ms/step - loss: 1.5779 - accuracy: 0.3182 - val_loss: 1.7499 - val_accuracy: 0.0091
Epoch 2/50
9/9 [==============================] - 0s 5ms/step - loss: 1.5245 - accuracy: 0.4727 - val_loss: 1.8824 - val_accuracy: 0.0000e+00
Epoch 3/50
9/9 [==============================] - 0s 5ms/step - loss: 1.4581 - accuracy: 0.5455 - val_loss: 2.0298 - val_accuracy: 0.0000e+00
Epoch 4/50
9/9 [==============================] - 0s 6ms/step - loss: 1.3751 - accuracy: 0.5659 - val_loss: 2.1834 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.490909	0.313253	0.373522	0.256383

Tuned_ANN(X_tfidf_fullset, y_tfidf_fullset)

4/4 [==============================] - 1s 6ms/step - loss: 1.6073 - accuracy: 0.1789
1/1 [==============================] - 0s 213ms/step - loss: 1.5968 - accuracy: 0.2857
4/4 [==============================] - 1s 4ms/step - loss: 1.5999 - accuracy: 0.2568
1/1 [==============================] - 0s 129ms/step - loss: 1.5885 - accuracy: 0.1707
4/4 [==============================] - 0s 4ms/step - loss: 1.6462 - accuracy: 0.1568
1/1 [==============================] - 0s 133ms/step - loss: 1.6358 - accuracy: 0.1951
4/4 [==============================] - 0s 3ms/step - loss: 1.6124 - accuracy: 0.2054
1/1 [==============================] - 0s 138ms/step - loss: 1.5846 - accuracy: 0.3902
4/4 [==============================] - 0s 3ms/step - loss: 1.6316 - accuracy: 0.1459
1/1 [==============================] - 0s 128ms/step - loss: 1.6118 - accuracy: 0.2683
4/4 [==============================] - 0s 3ms/step - loss: 1.6225 - accuracy: 0.2432
1/1 [==============================] - 0s 125ms/step - loss: 1.5945 - accuracy: 0.2439
4/4 [==============================] - 0s 3ms/step - loss: 1.5647 - accuracy: 0.3378
1/1 [==============================] - 0s 130ms/step - loss: 1.5485 - accuracy: 0.2927
4/4 [==============================] - 0s 3ms/step - loss: 1.6257 - accuracy: 0.2405
1/1 [==============================] - 0s 139ms/step - loss: 1.5565 - accuracy: 0.3171
4/4 [==============================] - 0s 3ms/step - loss: 1.6139 - accuracy: 0.1946
1/1 [==============================] - 0s 132ms/step - loss: 1.5948 - accuracy: 0.2927
4/4 [==============================] - 0s 3ms/step - loss: 1.5746 - accuracy: 0.3216
1/1 [==============================] - 0s 136ms/step - loss: 1.5569 - accuracy: 0.3171
4/4 [==============================] - 0s 4ms/step - loss: 1.5969 - accuracy: 0.2737
1/1 [==============================] - 0s 157ms/step - loss: 1.5235 - accuracy: 0.3095
4/4 [==============================] - 0s 3ms/step - loss: 1.5750 - accuracy: 0.3054
1/1 [==============================] - 0s 125ms/step - loss: 1.5327 - accuracy: 0.3659
4/4 [==============================] - 0s 4ms/step - loss: 1.6021 - accuracy: 0.1919
1/1 [==============================] - 0s 125ms/step - loss: 1.5563 - accuracy: 0.3415
4/4 [==============================] - 0s 4ms/step - loss: 1.6353 - accuracy: 0.1351
1/1 [==============================] - 0s 126ms/step - loss: 1.5859 - accuracy: 0.3171
4/4 [==============================] - 0s 4ms/step - loss: 1.6050 - accuracy: 0.2811
1/1 [==============================] - 0s 133ms/step - loss: 1.5807 - accuracy: 0.2195
4/4 [==============================] - 0s 7ms/step - loss: 1.6121 - accuracy: 0.2216
1/1 [==============================] - 0s 193ms/step - loss: 1.5931 - accuracy: 0.2195
4/4 [==============================] - 1s 5ms/step - loss: 1.5798 - accuracy: 0.2649
1/1 [==============================] - 0s 184ms/step - loss: 1.5434 - accuracy: 0.2927
4/4 [==============================] - 1s 6ms/step - loss: 1.5937 - accuracy: 0.2432
1/1 [==============================] - 0s 208ms/step - loss: 1.5623 - accuracy: 0.3659
4/4 [==============================] - 1s 4ms/step - loss: 1.5856 - accuracy: 0.2946
1/1 [==============================] - 0s 129ms/step - loss: 1.5952 - accuracy: 0.2439
4/4 [==============================] - 0s 4ms/step - loss: 1.5907 - accuracy: 0.2622
1/1 [==============================] - 0s 125ms/step - loss: 1.5383 - accuracy: 0.3659
4/4 [==============================] - 0s 5ms/step - loss: 1.5812 - accuracy: 0.2575
1/1 [==============================] - 0s 141ms/step - loss: 1.5075 - accuracy: 0.3571
4/4 [==============================] - 0s 4ms/step - loss: 1.5448 - accuracy: 0.3054
1/1 [==============================] - 0s 130ms/step - loss: 1.5187 - accuracy: 0.3171
4/4 [==============================] - 0s 4ms/step - loss: 1.5885 - accuracy: 0.2865
1/1 [==============================] - 0s 127ms/step - loss: 1.4824 - accuracy: 0.3902
4/4 [==============================] - 0s 4ms/step - loss: 1.5556 - accuracy: 0.2703
1/1 [==============================] - 0s 150ms/step - loss: 1.5706 - accuracy: 0.2195
4/4 [==============================] - 0s 4ms/step - loss: 1.5785 - accuracy: 0.2378
1/1 [==============================] - 0s 149ms/step - loss: 1.4404 - accuracy: 0.3659
4/4 [==============================] - 1s 4ms/step - loss: 1.5828 - accuracy: 0.2568
1/1 [==============================] - 0s 136ms/step - loss: 1.4865 - accuracy: 0.3171
4/4 [==============================] - 0s 6ms/step - loss: 1.5817 - accuracy: 0.2865
1/1 [==============================] - 0s 128ms/step - loss: 1.4996 - accuracy: 0.3171
4/4 [==============================] - 0s 4ms/step - loss: 1.5564 - accuracy: 0.3189
1/1 [==============================] - 0s 130ms/step - loss: 1.5666 - accuracy: 0.2927
4/4 [==============================] - 0s 4ms/step - loss: 1.5502 - accuracy: 0.3405
1/1 [==============================] - 0s 122ms/step - loss: 1.5675 - accuracy: 0.2927
4/4 [==============================] - 0s 4ms/step - loss: 1.6084 - accuracy: 0.2432
1/1 [==============================] - 0s 140ms/step - loss: 1.5334 - accuracy: 0.3902
4/4 [==============================] - 0s 5ms/step - loss: 1.5481 - accuracy: 0.2602
1/1 [==============================] - 0s 144ms/step - loss: 1.4511 - accuracy: 0.3333
4/4 [==============================] - 1s 7ms/step - loss: 1.5333 - accuracy: 0.3189
1/1 [==============================] - 0s 201ms/step - loss: 1.5088 - accuracy: 0.3415
4/4 [==============================] - 1s 7ms/step - loss: 1.5535 - accuracy: 0.3351
1/1 [==============================] - 0s 200ms/step - loss: 1.4161 - accuracy: 0.3415
4/4 [==============================] - 1s 8ms/step - loss: 1.5330 - accuracy: 0.3405
1/1 [==============================] - 0s 191ms/step - loss: 1.5820 - accuracy: 0.2195
4/4 [==============================] - 0s 5ms/step - loss: 1.5462 - accuracy: 0.3054
1/1 [==============================] - 0s 155ms/step - loss: 1.4183 - accuracy: 0.3171
4/4 [==============================] - 0s 5ms/step - loss: 1.5381 - accuracy: 0.3703
1/1 [==============================] - 0s 133ms/step - loss: 1.4005 - accuracy: 0.4878
4/4 [==============================] - 0s 5ms/step - loss: 1.5525 - accuracy: 0.3622
1/1 [==============================] - 0s 136ms/step - loss: 1.4431 - accuracy: 0.3659
4/4 [==============================] - 0s 5ms/step - loss: 1.5673 - accuracy: 0.3081
1/1 [==============================] - 0s 139ms/step - loss: 1.5366 - accuracy: 0.2927
4/4 [==============================] - 0s 5ms/step - loss: 1.5508 - accuracy: 0.2973
1/1 [==============================] - 0s 130ms/step - loss: 1.5830 - accuracy: 0.1951
4/4 [==============================] - 1s 10ms/step - loss: 1.5297 - accuracy: 0.3324
1/1 [==============================] - 0s 391ms/step - loss: 1.4746 - accuracy: 0.3902
4/4 [==============================] - 1s 6ms/step - loss: 1.6008 - accuracy: 0.2846
1/1 [==============================] - 0s 327ms/step - loss: 1.5639 - accuracy: 0.4048
4/4 [==============================] - 1s 4ms/step - loss: 1.6587 - accuracy: 0.1378
1/1 [==============================] - 0s 206ms/step - loss: 1.6214 - accuracy: 0.1951
4/4 [==============================] - 1s 6ms/step - loss: 1.6245 - accuracy: 0.1622
1/1 [==============================] - 0s 228ms/step - loss: 1.6042 - accuracy: 0.1951
4/4 [==============================] - 2s 5ms/step - loss: 1.5950 - accuracy: 0.2486
1/1 [==============================] - 0s 401ms/step - loss: 1.5862 - accuracy: 0.2683
4/4 [==============================] - 1s 6ms/step - loss: 1.5940 - accuracy: 0.2838
1/1 [==============================] - 0s 256ms/step - loss: 1.5799 - accuracy: 0.3415
4/4 [==============================] - 1s 9ms/step - loss: 1.6230 - accuracy: 0.1703
1/1 [==============================] - 0s 251ms/step - loss: 1.6112 - accuracy: 0.1220
4/4 [==============================] - 1s 7ms/step - loss: 1.5680 - accuracy: 0.3162
1/1 [==============================] - 0s 234ms/step - loss: 1.5345 - accuracy: 0.3171
4/4 [==============================] - 1s 6ms/step - loss: 1.6264 - accuracy: 0.1270
1/1 [==============================] - 0s 322ms/step - loss: 1.6182 - accuracy: 0.2439
4/4 [==============================] - 1s 4ms/step - loss: 1.5836 - accuracy: 0.2919
1/1 [==============================] - 0s 223ms/step - loss: 1.6223 - accuracy: 0.2195
4/4 [==============================] - 1s 5ms/step - loss: 1.6021 - accuracy: 0.1919
1/1 [==============================] - 0s 217ms/step - loss: 1.5828 - accuracy: 0.1463
4/4 [==============================] - 1s 5ms/step - loss: 1.5842 - accuracy: 0.2520
1/1 [==============================] - 0s 224ms/step - loss: 1.5267 - accuracy: 0.2381
4/4 [==============================] - 1s 4ms/step - loss: 1.6201 - accuracy: 0.2351
1/1 [==============================] - 0s 123ms/step - loss: 1.6121 - accuracy: 0.2683
4/4 [==============================] - 1s 5ms/step - loss: 1.5876 - accuracy: 0.2892
1/1 [==============================] - 1s 553ms/step - loss: 1.5367 - accuracy: 0.3659
4/4 [==============================] - 1s 6ms/step - loss: 1.5974 - accuracy: 0.1865
1/1 [==============================] - 1s 520ms/step - loss: 1.6113 - accuracy: 0.2439
4/4 [==============================] - 1s 13ms/step - loss: 1.5753 - accuracy: 0.2378
1/1 [==============================] - 0s 425ms/step - loss: 1.4824 - accuracy: 0.4878
4/4 [==============================] - 1s 5ms/step - loss: 1.6268 - accuracy: 0.1730
1/1 [==============================] - 0s 447ms/step - loss: 1.5767 - accuracy: 0.2439
4/4 [==============================] - 2s 7ms/step - loss: 1.5933 - accuracy: 0.2865
1/1 [==============================] - 1s 727ms/step - loss: 1.5593 - accuracy: 0.3902
4/4 [==============================] - 2s 6ms/step - loss: 1.6509 - accuracy: 0.1243
1/1 [==============================] - 0s 317ms/step - loss: 1.5919 - accuracy: 0.2683
4/4 [==============================] - 3s 8ms/step - loss: 1.6328 - accuracy: 0.1595
1/1 [==============================] - 0s 415ms/step - loss: 1.5977 - accuracy: 0.3659
4/4 [==============================] - 1s 7ms/step - loss: 1.6322 - accuracy: 0.1486
1/1 [==============================] - 0s 225ms/step - loss: 1.5955 - accuracy: 0.2195
4/4 [==============================] - 1s 8ms/step - loss: 1.5511 - accuracy: 0.2575
1/1 [==============================] - 0s 210ms/step - loss: 1.4763 - accuracy: 0.4762
4/4 [==============================] - 1s 9ms/step - loss: 1.5628 - accuracy: 0.3054
1/1 [==============================] - 0s 217ms/step - loss: 1.5217 - accuracy: 0.3415
4/4 [==============================] - 1s 8ms/step - loss: 1.5515 - accuracy: 0.3135
1/1 [==============================] - 0s 221ms/step - loss: 1.4492 - accuracy: 0.4390
4/4 [==============================] - 1s 10ms/step - loss: 1.5443 - accuracy: 0.3243
1/1 [==============================] - 0s 202ms/step - loss: 1.6143 - accuracy: 0.2195
4/4 [==============================] - 1s 7ms/step - loss: 1.5717 - accuracy: 0.2459
1/1 [==============================] - 1s 580ms/step - loss: 1.4639 - accuracy: 0.3415
4/4 [==============================] - 0s 4ms/step - loss: 1.5774 - accuracy: 0.2514
1/1 [==============================] - 0s 148ms/step - loss: 1.4972 - accuracy: 0.4634
4/4 [==============================] - 0s 4ms/step - loss: 1.5884 - accuracy: 0.3108
1/1 [==============================] - 0s 136ms/step - loss: 1.5116 - accuracy: 0.2439
4/4 [==============================] - 0s 4ms/step - loss: 1.5501 - accuracy: 0.3432
1/1 [==============================] - 0s 220ms/step - loss: 1.5019 - accuracy: 0.3171
4/4 [==============================] - 1s 8ms/step - loss: 1.5753 - accuracy: 0.2568
1/1 [==============================] - 0s 449ms/step - loss: 1.5818 - accuracy: 0.2439
4/4 [==============================] - 1s 8ms/step - loss: 1.5686 - accuracy: 0.3324
1/1 [==============================] - 0s 305ms/step - loss: 1.4922 - accuracy: 0.3902
4/4 [==============================] - 1s 7ms/step - loss: 1.5405 - accuracy: 0.2818
1/1 [==============================] - 0s 229ms/step - loss: 1.4563 - accuracy: 0.3810
4/4 [==============================] - 1s 9ms/step - loss: 1.5258 - accuracy: 0.3297
1/1 [==============================] - 0s 240ms/step - loss: 1.4597 - accuracy: 0.3415
4/4 [==============================] - 1s 8ms/step - loss: 1.5482 - accuracy: 0.3027
1/1 [==============================] - 0s 238ms/step - loss: 1.4059 - accuracy: 0.4146
4/4 [==============================] - 1s 9ms/step - loss: 1.5630 - accuracy: 0.2919
1/1 [==============================] - 0s 222ms/step - loss: 1.5457 - accuracy: 0.2195
4/4 [==============================] - 1s 10ms/step - loss: 1.5851 - accuracy: 0.3027
1/1 [==============================] - 0s 242ms/step - loss: 1.4566 - accuracy: 0.3415
4/4 [==============================] - 1s 12ms/step - loss: 1.5558 - accuracy: 0.3000
1/1 [==============================] - 0s 384ms/step - loss: 1.4071 - accuracy: 0.4878
4/4 [==============================] - 1s 8ms/step - loss: 1.5556 - accuracy: 0.3243
1/1 [==============================] - 0s 466ms/step - loss: 1.4682 - accuracy: 0.3171
4/4 [==============================] - 2s 11ms/step - loss: 1.5521 - accuracy: 0.3216
1/1 [==============================] - 1s 607ms/step - loss: 1.5350 - accuracy: 0.3171
4/4 [==============================] - 1s 9ms/step - loss: 1.5662 - accuracy: 0.2595
1/1 [==============================] - 1s 641ms/step - loss: 1.5455 - accuracy: 0.2439
4/4 [==============================] - 1s 8ms/step - loss: 1.5853 - accuracy: 0.2514
1/1 [==============================] - 0s 222ms/step - loss: 1.4700 - accuracy: 0.3659
19/19 [==============================] - 1s 4ms/step - loss: 1.5778 - accuracy: 0.2602
3/3 [==============================] - 0s 5ms/step - loss: 1.5176 - accuracy: 0.4048
19/19 [==============================] - 1s 4ms/step - loss: 1.5445 - accuracy: 0.3216
3/3 [==============================] - 0s 5ms/step - loss: 1.4560 - accuracy: 0.3171
19/19 [==============================] - 1s 5ms/step - loss: 1.5718 - accuracy: 0.3405
3/3 [==============================] - 0s 5ms/step - loss: 1.5093 - accuracy: 0.3659
19/19 [==============================] - 2s 4ms/step - loss: 1.5640 - accuracy: 0.3378
3/3 [==============================] - 1s 5ms/step - loss: 1.5505 - accuracy: 0.2439
19/19 [==============================] - 1s 5ms/step - loss: 1.5249 - accuracy: 0.3622
3/3 [==============================] - 0s 10ms/step - loss: 1.4499 - accuracy: 0.3659
19/19 [==============================] - 1s 3ms/step - loss: 1.5787 - accuracy: 0.3027
3/3 [==============================] - 0s 3ms/step - loss: 1.5090 - accuracy: 0.2927
19/19 [==============================] - 1s 4ms/step - loss: 1.5350 - accuracy: 0.3243
3/3 [==============================] - 0s 3ms/step - loss: 1.4578 - accuracy: 0.4146
19/19 [==============================] - 1s 5ms/step - loss: 1.5764 - accuracy: 0.3432
3/3 [==============================] - 1s 4ms/step - loss: 1.5443 - accuracy: 0.3415
19/19 [==============================] - 1s 4ms/step - loss: 1.5531 - accuracy: 0.3216
3/3 [==============================] - 0s 7ms/step - loss: 1.5669 - accuracy: 0.2927
19/19 [==============================] - 1s 4ms/step - loss: 1.5695 - accuracy: 0.3135
3/3 [==============================] - 0s 2ms/step - loss: 1.5198 - accuracy: 0.3902
19/19 [==============================] - 1s 4ms/step - loss: 1.5367 - accuracy: 0.2791
3/3 [==============================] - 0s 8ms/step - loss: 1.4569 - accuracy: 0.4048
19/19 [==============================] - 1s 5ms/step - loss: 1.5353 - accuracy: 0.3703
3/3 [==============================] - 0s 6ms/step - loss: 1.5055 - accuracy: 0.2927
19/19 [==============================] - 1s 4ms/step - loss: 1.5192 - accuracy: 0.3243
3/3 [==============================] - 0s 5ms/step - loss: 1.3722 - accuracy: 0.4634
19/19 [==============================] - 1s 4ms/step - loss: 1.4977 - accuracy: 0.3459
3/3 [==============================] - 0s 3ms/step - loss: 1.5653 - accuracy: 0.2195
19/19 [==============================] - 1s 4ms/step - loss: 1.5425 - accuracy: 0.3297
3/3 [==============================] - 0s 4ms/step - loss: 1.4112 - accuracy: 0.3659
19/19 [==============================] - 1s 4ms/step - loss: 1.5214 - accuracy: 0.3405
3/3 [==============================] - 0s 3ms/step - loss: 1.4034 - accuracy: 0.3415
19/19 [==============================] - 1s 2ms/step - loss: 1.5090 - accuracy: 0.3216
3/3 [==============================] - 0s 3ms/step - loss: 1.4051 - accuracy: 0.3171
19/19 [==============================] - 1s 2ms/step - loss: 1.5287 - accuracy: 0.3297
3/3 [==============================] - 0s 5ms/step - loss: 1.5307 - accuracy: 0.2195
19/19 [==============================] - 1s 4ms/step - loss: 1.5402 - accuracy: 0.3027
3/3 [==============================] - 0s 6ms/step - loss: 1.5276 - accuracy: 0.2927
19/19 [==============================] - 1s 2ms/step - loss: 1.5122 - accuracy: 0.3324
3/3 [==============================] - 0s 5ms/step - loss: 1.4340 - accuracy: 0.3415
19/19 [==============================] - 1s 4ms/step - loss: 1.4824 - accuracy: 0.3008
3/3 [==============================] - 0s 5ms/step - loss: 1.4176 - accuracy: 0.4286
19/19 [==============================] - 1s 2ms/step - loss: 1.4662 - accuracy: 0.3514
3/3 [==============================] - 0s 6ms/step - loss: 1.4594 - accuracy: 0.3171
19/19 [==============================] - 2s 5ms/step - loss: 1.4822 - accuracy: 0.3838
3/3 [==============================] - 1s 10ms/step - loss: 1.3025 - accuracy: 0.3415
19/19 [==============================] - 1s 6ms/step - loss: 1.4684 - accuracy: 0.3405
3/3 [==============================] - 0s 7ms/step - loss: 1.6228 - accuracy: 0.2195
19/19 [==============================] - 1s 2ms/step - loss: 1.4832 - accuracy: 0.3405
3/3 [==============================] - 0s 6ms/step - loss: 1.3263 - accuracy: 0.3415
19/19 [==============================] - 1s 2ms/step - loss: 1.4945 - accuracy: 0.3054
3/3 [==============================] - 0s 4ms/step - loss: 1.3198 - accuracy: 0.4146
19/19 [==============================] - 1s 2ms/step - loss: 1.4567 - accuracy: 0.3486
3/3 [==============================] - 0s 6ms/step - loss: 1.3823 - accuracy: 0.2927
19/19 [==============================] - 1s 4ms/step - loss: 1.4712 - accuracy: 0.3595
3/3 [==============================] - 0s 7ms/step - loss: 1.4989 - accuracy: 0.2683
19/19 [==============================] - 1s 4ms/step - loss: 1.4861 - accuracy: 0.3378
3/3 [==============================] - 0s 6ms/step - loss: 1.5926 - accuracy: 0.2683
19/19 [==============================] - 1s 4ms/step - loss: 1.4987 - accuracy: 0.3054
3/3 [==============================] - 0s 4ms/step - loss: 1.4305 - accuracy: 0.4146
19/19 [==============================] - 1s 5ms/step - loss: 1.4641 - accuracy: 0.3089
3/3 [==============================] - 0s 7ms/step - loss: 1.3964 - accuracy: 0.5238
19/19 [==============================] - 1s 7ms/step - loss: 1.4556 - accuracy: 0.3459
3/3 [==============================] - 0s 8ms/step - loss: 1.4569 - accuracy: 0.2439
19/19 [==============================] - 1s 8ms/step - loss: 1.4500 - accuracy: 0.3486
3/3 [==============================] - 0s 7ms/step - loss: 1.2421 - accuracy: 0.3659
19/19 [==============================] - 1s 5ms/step - loss: 1.4343 - accuracy: 0.3486
3/3 [==============================] - 0s 7ms/step - loss: 1.5284 - accuracy: 0.2683
19/19 [==============================] - 1s 5ms/step - loss: 1.4509 - accuracy: 0.3568
3/3 [==============================] - 0s 6ms/step - loss: 1.2748 - accuracy: 0.3902
19/19 [==============================] - 1s 5ms/step - loss: 1.4582 - accuracy: 0.3378
3/3 [==============================] - 0s 10ms/step - loss: 1.2604 - accuracy: 0.4634
19/19 [==============================] - 1s 5ms/step - loss: 1.4540 - accuracy: 0.3459
3/3 [==============================] - 0s 7ms/step - loss: 1.3723 - accuracy: 0.2927
19/19 [==============================] - 1s 5ms/step - loss: 1.4290 - accuracy: 0.3865
3/3 [==============================] - 0s 4ms/step - loss: 1.4772 - accuracy: 0.3415
19/19 [==============================] - 0s 3ms/step - loss: 1.4500 - accuracy: 0.3703
3/3 [==============================] - 0s 4ms/step - loss: 1.5701 - accuracy: 0.2683
19/19 [==============================] - 0s 3ms/step - loss: 1.4884 - accuracy: 0.3000
3/3 [==============================] - 0s 4ms/step - loss: 1.4016 - accuracy: 0.3659
19/19 [==============================] - 1s 4ms/step - loss: 1.5496 - accuracy: 0.3171
3/3 [==============================] - 0s 7ms/step - loss: 1.4824 - accuracy: 0.3095
19/19 [==============================] - 1s 4ms/step - loss: 1.5601 - accuracy: 0.3270
3/3 [==============================] - 0s 6ms/step - loss: 1.5458 - accuracy: 0.3415
19/19 [==============================] - 1s 5ms/step - loss: 1.5787 - accuracy: 0.2676
3/3 [==============================] - 0s 5ms/step - loss: 1.5141 - accuracy: 0.3902
19/19 [==============================] - 1s 3ms/step - loss: 1.5799 - accuracy: 0.2595
3/3 [==============================] - 1s 5ms/step - loss: 1.6070 - accuracy: 0.1463
19/19 [==============================] - 1s 4ms/step - loss: 1.5850 - accuracy: 0.3027
3/3 [==============================] - 0s 3ms/step - loss: 1.5220 - accuracy: 0.3171
19/19 [==============================] - 1s 3ms/step - loss: 1.5432 - accuracy: 0.3541
3/3 [==============================] - 0s 5ms/step - loss: 1.4271 - accuracy: 0.4878
19/19 [==============================] - 1s 2ms/step - loss: 1.5741 - accuracy: 0.2568
3/3 [==============================] - 0s 3ms/step - loss: 1.4736 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5760 - accuracy: 0.2703
3/3 [==============================] - 0s 4ms/step - loss: 1.5826 - accuracy: 0.1951
19/19 [==============================] - 0s 2ms/step - loss: 1.5843 - accuracy: 0.2514
3/3 [==============================] - 0s 4ms/step - loss: 1.5605 - accuracy: 0.2195
19/19 [==============================] - 1s 5ms/step - loss: 1.5562 - accuracy: 0.3108
3/3 [==============================] - 0s 5ms/step - loss: 1.4856 - accuracy: 0.3415
19/19 [==============================] - 1s 4ms/step - loss: 1.5093 - accuracy: 0.3333
3/3 [==============================] - 0s 4ms/step - loss: 1.4389 - accuracy: 0.4048
19/19 [==============================] - 1s 5ms/step - loss: 1.5347 - accuracy: 0.3351
3/3 [==============================] - 0s 3ms/step - loss: 1.4907 - accuracy: 0.3415
19/19 [==============================] - 1s 4ms/step - loss: 1.5345 - accuracy: 0.3514
3/3 [==============================] - 0s 7ms/step - loss: 1.4127 - accuracy: 0.3659
19/19 [==============================] - 1s 4ms/step - loss: 1.5037 - accuracy: 0.3324
3/3 [==============================] - 0s 7ms/step - loss: 1.5576 - accuracy: 0.2927
19/19 [==============================] - 1s 4ms/step - loss: 1.5241 - accuracy: 0.2865
3/3 [==============================] - 0s 7ms/step - loss: 1.3843 - accuracy: 0.3415
19/19 [==============================] - 1s 5ms/step - loss: 1.5533 - accuracy: 0.1919
3/3 [==============================] - 0s 8ms/step - loss: 1.4356 - accuracy: 0.3171
19/19 [==============================] - 1s 4ms/step - loss: 1.5442 - accuracy: 0.3081
3/3 [==============================] - 0s 3ms/step - loss: 1.4537 - accuracy: 0.3171
19/19 [==============================] - 1s 7ms/step - loss: 1.5412 - accuracy: 0.2541
3/3 [==============================] - 0s 7ms/step - loss: 1.5625 - accuracy: 0.1951
19/19 [==============================] - 1s 5ms/step - loss: 1.5286 - accuracy: 0.3216
3/3 [==============================] - 1s 4ms/step - loss: 1.5848 - accuracy: 0.3415
19/19 [==============================] - 1s 5ms/step - loss: 1.5314 - accuracy: 0.3135
3/3 [==============================] - 0s 4ms/step - loss: 1.4409 - accuracy: 0.3902
19/19 [==============================] - 1s 5ms/step - loss: 1.5216 - accuracy: 0.2900
3/3 [==============================] - 0s 7ms/step - loss: 1.4287 - accuracy: 0.3810
19/19 [==============================] - 1s 5ms/step - loss: 1.4680 - accuracy: 0.3730
3/3 [==============================] - 0s 7ms/step - loss: 1.4476 - accuracy: 0.3171
19/19 [==============================] - 1s 4ms/step - loss: 1.5065 - accuracy: 0.3514
3/3 [==============================] - 0s 6ms/step - loss: 1.3140 - accuracy: 0.3902
19/19 [==============================] - 1s 5ms/step - loss: 1.4689 - accuracy: 0.3216
3/3 [==============================] - 0s 3ms/step - loss: 1.5604 - accuracy: 0.2195
19/19 [==============================] - 1s 4ms/step - loss: 1.4667 - accuracy: 0.3568
3/3 [==============================] - 0s 6ms/step - loss: 1.3216 - accuracy: 0.3415
19/19 [==============================] - 1s 4ms/step - loss: 1.4651 - accuracy: 0.3676
3/3 [==============================] - 0s 3ms/step - loss: 1.2792 - accuracy: 0.4878
19/19 [==============================] - 1s 4ms/step - loss: 1.4830 - accuracy: 0.3351
3/3 [==============================] - 0s 5ms/step - loss: 1.3686 - accuracy: 0.2927
19/19 [==============================] - 1s 5ms/step - loss: 1.4925 - accuracy: 0.3135
3/3 [==============================] - 0s 4ms/step - loss: 1.4910 - accuracy: 0.2927
19/19 [==============================] - 1s 5ms/step - loss: 1.4809 - accuracy: 0.3514
3/3 [==============================] - 0s 3ms/step - loss: 1.5558 - accuracy: 0.2927
19/19 [==============================] - 1s 4ms/step - loss: 1.5127 - accuracy: 0.3081
3/3 [==============================] - 0s 6ms/step - loss: 1.4391 - accuracy: 0.3902
19/19 [==============================] - 1s 5ms/step - loss: 1.4798 - accuracy: 0.2846
3/3 [==============================] - 0s 7ms/step - loss: 1.4079 - accuracy: 0.3810
19/19 [==============================] - 1s 5ms/step - loss: 1.4619 - accuracy: 0.3541
3/3 [==============================] - 0s 3ms/step - loss: 1.4547 - accuracy: 0.2195
19/19 [==============================] - 1s 5ms/step - loss: 1.4501 - accuracy: 0.3514
3/3 [==============================] - 0s 5ms/step - loss: 1.2558 - accuracy: 0.3659
19/19 [==============================] - 1s 5ms/step - loss: 1.4452 - accuracy: 0.3405
3/3 [==============================] - 0s 7ms/step - loss: 1.5902 - accuracy: 0.2195
19/19 [==============================] - 1s 5ms/step - loss: 1.4709 - accuracy: 0.3243
3/3 [==============================] - 0s 6ms/step - loss: 1.2682 - accuracy: 0.3171
19/19 [==============================] - 1s 7ms/step - loss: 1.4547 - accuracy: 0.3351
3/3 [==============================] - 1s 7ms/step - loss: 1.2690 - accuracy: 0.4390
19/19 [==============================] - 3s 5ms/step - loss: 1.4472 - accuracy: 0.3757
3/3 [==============================] - 0s 3ms/step - loss: 1.3375 - accuracy: 0.2927
19/19 [==============================] - 1s 3ms/step - loss: 1.4502 - accuracy: 0.3892
3/3 [==============================] - 0s 4ms/step - loss: 1.5113 - accuracy: 0.3171
19/19 [==============================] - 0s 3ms/step - loss: 1.4534 - accuracy: 0.3568
3/3 [==============================] - 0s 3ms/step - loss: 1.5676 - accuracy: 0.2683
19/19 [==============================] - 0s 4ms/step - loss: 1.4646 - accuracy: 0.3351
3/3 [==============================] - 0s 4ms/step - loss: 1.3942 - accuracy: 0.3902
8/8 [==============================] - 0s 3ms/step - loss: 1.6207 - accuracy: 0.1572
1/1 [==============================] - 0s 138ms/step - loss: 1.5687 - accuracy: 0.3333
8/8 [==============================] - 1s 6ms/step - loss: 1.5693 - accuracy: 0.3135
1/1 [==============================] - 0s 247ms/step - loss: 1.5618 - accuracy: 0.3415
8/8 [==============================] - 1s 5ms/step - loss: 1.5793 - accuracy: 0.2865
1/1 [==============================] - 0s 221ms/step - loss: 1.5423 - accuracy: 0.3415
8/8 [==============================] - 1s 4ms/step - loss: 1.5496 - accuracy: 0.3027
1/1 [==============================] - 0s 208ms/step - loss: 1.5851 - accuracy: 0.2683
8/8 [==============================] - 1s 3ms/step - loss: 1.6012 - accuracy: 0.2946
1/1 [==============================] - 0s 130ms/step - loss: 1.5495 - accuracy: 0.4146
8/8 [==============================] - 0s 4ms/step - loss: 1.6063 - accuracy: 0.2135
1/1 [==============================] - 0s 186ms/step - loss: 1.5435 - accuracy: 0.2439
8/8 [==============================] - 1s 3ms/step - loss: 1.5780 - accuracy: 0.2919
1/1 [==============================] - 0s 252ms/step - loss: 1.5436 - accuracy: 0.2683
8/8 [==============================] - 1s 4ms/step - loss: 1.5601 - accuracy: 0.3378
1/1 [==============================] - 0s 217ms/step - loss: 1.5565 - accuracy: 0.3171
8/8 [==============================] - 1s 4ms/step - loss: 1.5743 - accuracy: 0.2973
1/1 [==============================] - 0s 260ms/step - loss: 1.6022 - accuracy: 0.2195
8/8 [==============================] - 1s 4ms/step - loss: 1.6340 - accuracy: 0.1351
1/1 [==============================] - 0s 217ms/step - loss: 1.6027 - accuracy: 0.2439
8/8 [==============================] - 1s 3ms/step - loss: 1.5970 - accuracy: 0.2168
1/1 [==============================] - 0s 140ms/step - loss: 1.5308 - accuracy: 0.2619
8/8 [==============================] - 0s 3ms/step - loss: 1.5804 - accuracy: 0.2892
1/1 [==============================] - 0s 126ms/step - loss: 1.5358 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.5755 - accuracy: 0.3054
1/1 [==============================] - 0s 132ms/step - loss: 1.4997 - accuracy: 0.3171
8/8 [==============================] - 0s 3ms/step - loss: 1.5974 - accuracy: 0.2297
1/1 [==============================] - 0s 143ms/step - loss: 1.5603 - accuracy: 0.3902
8/8 [==============================] - 0s 3ms/step - loss: 1.5501 - accuracy: 0.3568
1/1 [==============================] - 0s 127ms/step - loss: 1.4699 - accuracy: 0.3171
8/8 [==============================] - 0s 3ms/step - loss: 1.5573 - accuracy: 0.2811
1/1 [==============================] - 0s 138ms/step - loss: 1.4825 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.5450 - accuracy: 0.2946
1/1 [==============================] - 0s 132ms/step - loss: 1.4880 - accuracy: 0.2439
8/8 [==============================] - 0s 4ms/step - loss: 1.5545 - accuracy: 0.2676
1/1 [==============================] - 0s 196ms/step - loss: 1.5405 - accuracy: 0.2683
8/8 [==============================] - 1s 5ms/step - loss: 1.5716 - accuracy: 0.2108
1/1 [==============================] - 0s 287ms/step - loss: 1.5822 - accuracy: 0.1463
8/8 [==============================] - 1s 5ms/step - loss: 1.5997 - accuracy: 0.2054
1/1 [==============================] - 0s 282ms/step - loss: 1.5344 - accuracy: 0.3902
8/8 [==============================] - 1s 8ms/step - loss: 1.5291 - accuracy: 0.3035
1/1 [==============================] - 1s 548ms/step - loss: 1.4522 - accuracy: 0.3571
8/8 [==============================] - 2s 10ms/step - loss: 1.5373 - accuracy: 0.3432
1/1 [==============================] - 0s 302ms/step - loss: 1.4821 - accuracy: 0.2927
8/8 [==============================] - 1s 3ms/step - loss: 1.5224 - accuracy: 0.3135
1/1 [==============================] - 0s 162ms/step - loss: 1.3702 - accuracy: 0.3902
8/8 [==============================] - 1s 5ms/step - loss: 1.5304 - accuracy: 0.3351
1/1 [==============================] - 0s 220ms/step - loss: 1.5640 - accuracy: 0.2683
8/8 [==============================] - 1s 6ms/step - loss: 1.5327 - accuracy: 0.3054
1/1 [==============================] - 0s 224ms/step - loss: 1.3925 - accuracy: 0.2927
8/8 [==============================] - 1s 5ms/step - loss: 1.5605 - accuracy: 0.2568
1/1 [==============================] - 0s 205ms/step - loss: 1.4262 - accuracy: 0.4146
8/8 [==============================] - 1s 6ms/step - loss: 1.5597 - accuracy: 0.3243
1/1 [==============================] - 0s 219ms/step - loss: 1.4560 - accuracy: 0.3415
8/8 [==============================] - 1s 5ms/step - loss: 1.5658 - accuracy: 0.3027
1/1 [==============================] - 0s 222ms/step - loss: 1.5388 - accuracy: 0.2927
8/8 [==============================] - 1s 6ms/step - loss: 1.5502 - accuracy: 0.3081
1/1 [==============================] - 0s 222ms/step - loss: 1.5918 - accuracy: 0.2683
8/8 [==============================] - 1s 5ms/step - loss: 1.5373 - accuracy: 0.2838
1/1 [==============================] - 0s 203ms/step - loss: 1.4492 - accuracy: 0.3659
8/8 [==============================] - 1s 6ms/step - loss: 1.5099 - accuracy: 0.3035
1/1 [==============================] - 0s 178ms/step - loss: 1.4225 - accuracy: 0.3810
8/8 [==============================] - 1s 7ms/step - loss: 1.5083 - accuracy: 0.3189
1/1 [==============================] - 0s 202ms/step - loss: 1.4803 - accuracy: 0.3415
8/8 [==============================] - 0s 4ms/step - loss: 1.5013 - accuracy: 0.3568
1/1 [==============================] - 0s 143ms/step - loss: 1.3259 - accuracy: 0.3659
8/8 [==============================] - 0s 4ms/step - loss: 1.4848 - accuracy: 0.3405
1/1 [==============================] - 0s 134ms/step - loss: 1.5859 - accuracy: 0.2195
8/8 [==============================] - 1s 7ms/step - loss: 1.5219 - accuracy: 0.3162
1/1 [==============================] - 0s 236ms/step - loss: 1.3544 - accuracy: 0.3415
8/8 [==============================] - 1s 7ms/step - loss: 1.5200 - accuracy: 0.2919
1/1 [==============================] - 0s 225ms/step - loss: 1.3554 - accuracy: 0.4634
8/8 [==============================] - 1s 8ms/step - loss: 1.4932 - accuracy: 0.3486
1/1 [==============================] - 0s 348ms/step - loss: 1.3806 - accuracy: 0.2927
8/8 [==============================] - 1s 7ms/step - loss: 1.5024 - accuracy: 0.3324
1/1 [==============================] - 0s 365ms/step - loss: 1.4902 - accuracy: 0.2439
8/8 [==============================] - 1s 8ms/step - loss: 1.5295 - accuracy: 0.2757
1/1 [==============================] - 0s 341ms/step - loss: 1.6137 - accuracy: 0.2683
8/8 [==============================] - 2s 11ms/step - loss: 1.5180 - accuracy: 0.2838
1/1 [==============================] - 1s 669ms/step - loss: 1.4149 - accuracy: 0.4146
8/8 [==============================] - 1s 3ms/step - loss: 1.5479 - accuracy: 0.3279
1/1 [==============================] - 0s 237ms/step - loss: 1.5197 - accuracy: 0.3810
8/8 [==============================] - 1s 4ms/step - loss: 1.5970 - accuracy: 0.2730
1/1 [==============================] - 0s 231ms/step - loss: 1.5699 - accuracy: 0.3171
8/8 [==============================] - 1s 4ms/step - loss: 1.5602 - accuracy: 0.2811
1/1 [==============================] - 0s 299ms/step - loss: 1.5092 - accuracy: 0.3659
8/8 [==============================] - 1s 5ms/step - loss: 1.5962 - accuracy: 0.2865
1/1 [==============================] - 0s 318ms/step - loss: 1.5955 - accuracy: 0.3415
8/8 [==============================] - 1s 4ms/step - loss: 1.5889 - accuracy: 0.2459
1/1 [==============================] - 0s 332ms/step - loss: 1.5662 - accuracy: 0.2683
8/8 [==============================] - 1s 3ms/step - loss: 1.5659 - accuracy: 0.3297
1/1 [==============================] - 0s 235ms/step - loss: 1.4802 - accuracy: 0.4146
8/8 [==============================] - 1s 5ms/step - loss: 1.6027 - accuracy: 0.2622
1/1 [==============================] - 0s 234ms/step - loss: 1.5372 - accuracy: 0.3659
8/8 [==============================] - 2s 8ms/step - loss: 1.6229 - accuracy: 0.1378
1/1 [==============================] - 1s 552ms/step - loss: 1.6130 - accuracy: 0.1463
8/8 [==============================] - 1s 5ms/step - loss: 1.5901 - accuracy: 0.2297
1/1 [==============================] - 0s 276ms/step - loss: 1.6017 - accuracy: 0.2439
8/8 [==============================] - 1s 5ms/step - loss: 1.5557 - accuracy: 0.3514
1/1 [==============================] - 0s 313ms/step - loss: 1.5327 - accuracy: 0.3659
8/8 [==============================] - 1s 5ms/step - loss: 1.5850 - accuracy: 0.2358
1/1 [==============================] - 0s 224ms/step - loss: 1.5438 - accuracy: 0.3571
8/8 [==============================] - 1s 6ms/step - loss: 1.5450 - accuracy: 0.3405
1/1 [==============================] - 0s 238ms/step - loss: 1.4991 - accuracy: 0.3415
8/8 [==============================] - 1s 5ms/step - loss: 1.5776 - accuracy: 0.2649
1/1 [==============================] - 0s 301ms/step - loss: 1.4900 - accuracy: 0.3902
8/8 [==============================] - 1s 5ms/step - loss: 1.5635 - accuracy: 0.2622
1/1 [==============================] - 0s 348ms/step - loss: 1.5706 - accuracy: 0.2195
8/8 [==============================] - 1s 12ms/step - loss: 1.5544 - accuracy: 0.3432
1/1 [==============================] - 1s 563ms/step - loss: 1.4687 - accuracy: 0.3659
8/8 [==============================] - 1s 5ms/step - loss: 1.5729 - accuracy: 0.2838
1/1 [==============================] - 0s 288ms/step - loss: 1.4949 - accuracy: 0.4390
8/8 [==============================] - 1s 5ms/step - loss: 1.5706 - accuracy: 0.2622
1/1 [==============================] - 0s 274ms/step - loss: 1.4912 - accuracy: 0.3171
8/8 [==============================] - 1s 6ms/step - loss: 1.5707 - accuracy: 0.3162
1/1 [==============================] - 0s 231ms/step - loss: 1.5465 - accuracy: 0.3415
8/8 [==============================] - 1s 5ms/step - loss: 1.5872 - accuracy: 0.2081
1/1 [==============================] - 0s 238ms/step - loss: 1.5766 - accuracy: 0.2927
8/8 [==============================] - 1s 5ms/step - loss: 1.5335 - accuracy: 0.3027
1/1 [==============================] - 0s 224ms/step - loss: 1.4638 - accuracy: 0.4634
8/8 [==============================] - 1s 7ms/step - loss: 1.5244 - accuracy: 0.3225
1/1 [==============================] - 0s 214ms/step - loss: 1.4482 - accuracy: 0.3571
8/8 [==============================] - 1s 6ms/step - loss: 1.5163 - accuracy: 0.3270
1/1 [==============================] - 0s 207ms/step - loss: 1.4871 - accuracy: 0.2927
8/8 [==============================] - 1s 3ms/step - loss: 1.5331 - accuracy: 0.3378
1/1 [==============================] - 0s 141ms/step - loss: 1.4117 - accuracy: 0.3902
8/8 [==============================] - 0s 4ms/step - loss: 1.5417 - accuracy: 0.3270
1/1 [==============================] - 0s 123ms/step - loss: 1.5688 - accuracy: 0.2439
8/8 [==============================] - 1s 4ms/step - loss: 1.5327 - accuracy: 0.3216
1/1 [==============================] - 0s 197ms/step - loss: 1.3814 - accuracy: 0.3415
8/8 [==============================] - 1s 5ms/step - loss: 1.5573 - accuracy: 0.2649
1/1 [==============================] - 0s 184ms/step - loss: 1.3935 - accuracy: 0.4634
8/8 [==============================] - 1s 5ms/step - loss: 1.5482 - accuracy: 0.3189
1/1 [==============================] - 0s 238ms/step - loss: 1.4421 - accuracy: 0.3171
8/8 [==============================] - 0s 4ms/step - loss: 1.5327 - accuracy: 0.3162
1/1 [==============================] - 0s 151ms/step - loss: 1.5222 - accuracy: 0.2683
8/8 [==============================] - 0s 3ms/step - loss: 1.5794 - accuracy: 0.2784
1/1 [==============================] - 0s 134ms/step - loss: 1.5471 - accuracy: 0.2439
8/8 [==============================] - 0s 3ms/step - loss: 1.5477 - accuracy: 0.3270
1/1 [==============================] - 0s 126ms/step - loss: 1.4465 - accuracy: 0.4146
8/8 [==============================] - 0s 4ms/step - loss: 1.5400 - accuracy: 0.2493
1/1 [==============================] - 0s 143ms/step - loss: 1.4359 - accuracy: 0.3571
8/8 [==============================] - 0s 4ms/step - loss: 1.5019 - accuracy: 0.3054
1/1 [==============================] - 0s 130ms/step - loss: 1.4582 - accuracy: 0.3415
8/8 [==============================] - 0s 4ms/step - loss: 1.5117 - accuracy: 0.3081
1/1 [==============================] - 0s 133ms/step - loss: 1.3273 - accuracy: 0.3902
8/8 [==============================] - 0s 4ms/step - loss: 1.5234 - accuracy: 0.3297
1/1 [==============================] - 0s 122ms/step - loss: 1.6031 - accuracy: 0.2195
8/8 [==============================] - 0s 4ms/step - loss: 1.5241 - accuracy: 0.3027
1/1 [==============================] - 0s 138ms/step - loss: 1.3435 - accuracy: 0.3902
8/8 [==============================] - 0s 5ms/step - loss: 1.5132 - accuracy: 0.3162
1/1 [==============================] - 0s 129ms/step - loss: 1.3468 - accuracy: 0.4634
8/8 [==============================] - 0s 4ms/step - loss: 1.5301 - accuracy: 0.3027
1/1 [==============================] - 0s 130ms/step - loss: 1.4127 - accuracy: 0.3171
8/8 [==============================] - 0s 4ms/step - loss: 1.5019 - accuracy: 0.3568
1/1 [==============================] - 0s 132ms/step - loss: 1.4878 - accuracy: 0.2683
8/8 [==============================] - 0s 4ms/step - loss: 1.5020 - accuracy: 0.3216
1/1 [==============================] - 0s 137ms/step - loss: 1.6231 - accuracy: 0.2683
8/8 [==============================] - 0s 5ms/step - loss: 1.5174 - accuracy: 0.2919
1/1 [==============================] - 0s 133ms/step - loss: 1.4520 - accuracy: 0.3659
9/9 [==============================] - 1s 4ms/step - loss: 1.5702 - accuracy: 0.2822
best parameters for ANN:  {'batch_size': 50, 'nb_epoch': 50, 'unit': 100}
best score for ANN: 0.3527874559164047

Best Parameters for TFIDF dataset-

best parameters for ANN: {'batch_size': 50, 'nb_epoch': 50, 'unit': 100}

best score for ANN: 0.3527874559164047

def NN_Model_Tuned_TFIDFFull(X_train, X_test, y_train, y_test):
  in_dim = X_train.shape[1]

  tf.random.set_seed(7)

  model = Sequential()
  model.add(Dense(50, activation='relu', input_dim = in_dim))
  model.add(Dropout(0.2))
  model.add(Dense(50, activation='relu'))
  model.add(Dense(5 , activation='softmax'))

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=0, patience=3)

  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

  print(model.summary())

  history=model.fit(X_train, y_train_cat, validation_split=0.2, epochs = 50, batch_size = 50, callbacks=[early_stopping])

  train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  result_kfold_df= pd.DataFrame({'model': ['Neural Network'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })



  hist= pd.DataFrame(history.history)

  for col in hist.columns:
    print(col)
    plt.plot(hist[col])
    plt.plot(hist[col])
    plt.title('model-'+col)
    plt.ylabel(col)
    plt.xlabel('epoch')
    plt.show()

  return result_kfold_df

NN_Model_Tuned_TFIDFFull(X_train_tfidffull, X_test_tfidffull, y_train_tfidffull, y_test_tfidffull)

Model: "sequential_731"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_2199 (Dense)          (None, 50)                11000     
                                                                 
 dropout_10 (Dropout)        (None, 50)                0         
                                                                 
 dense_2200 (Dense)          (None, 50)                2550      
                                                                 
 dense_2201 (Dense)          (None, 5)                 255       
                                                                 
=================================================================
Total params: 13,805
Trainable params: 13,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
6/6 [==============================] - 1s 75ms/step - loss: 1.6217 - accuracy: 0.1832 - val_loss: 1.6086 - val_accuracy: 0.1515
Epoch 2/50
6/6 [==============================] - 0s 12ms/step - loss: 1.5776 - accuracy: 0.3550 - val_loss: 1.5817 - val_accuracy: 0.3333
Epoch 3/50
6/6 [==============================] - 0s 13ms/step - loss: 1.5461 - accuracy: 0.3511 - val_loss: 1.5616 - val_accuracy: 0.3636
Epoch 4/50
6/6 [==============================] - 0s 12ms/step - loss: 1.5120 - accuracy: 0.4122 - val_loss: 1.5413 - val_accuracy: 0.3636
Epoch 5/50
6/6 [==============================] - 0s 12ms/step - loss: 1.4724 - accuracy: 0.4008 - val_loss: 1.5218 - val_accuracy: 0.3485
Epoch 6/50
6/6 [==============================] - 0s 19ms/step - loss: 1.4463 - accuracy: 0.4008 - val_loss: 1.5041 - val_accuracy: 0.3333
Epoch 7/50
6/6 [==============================] - 0s 15ms/step - loss: 1.4126 - accuracy: 0.3779 - val_loss: 1.4882 - val_accuracy: 0.3333
Epoch 8/50
6/6 [==============================] - 0s 19ms/step - loss: 1.3818 - accuracy: 0.3931 - val_loss: 1.4772 - val_accuracy: 0.3333
Epoch 9/50
6/6 [==============================] - 0s 11ms/step - loss: 1.3451 - accuracy: 0.3969 - val_loss: 1.4691 - val_accuracy: 0.3182
Epoch 10/50
6/6 [==============================] - 0s 17ms/step - loss: 1.3024 - accuracy: 0.4580 - val_loss: 1.4607 - val_accuracy: 0.3333
Epoch 11/50
6/6 [==============================] - 0s 15ms/step - loss: 1.2826 - accuracy: 0.5000 - val_loss: 1.4518 - val_accuracy: 0.3485
Epoch 12/50
6/6 [==============================] - 0s 12ms/step - loss: 1.2429 - accuracy: 0.5649 - val_loss: 1.4400 - val_accuracy: 0.3788
Epoch 13/50
6/6 [==============================] - 0s 11ms/step - loss: 1.2140 - accuracy: 0.5573 - val_loss: 1.4298 - val_accuracy: 0.3939
Epoch 14/50
6/6 [==============================] - 0s 14ms/step - loss: 1.1925 - accuracy: 0.5878 - val_loss: 1.4227 - val_accuracy: 0.4242
Epoch 15/50
6/6 [==============================] - 0s 13ms/step - loss: 1.1541 - accuracy: 0.5878 - val_loss: 1.4113 - val_accuracy: 0.4242
Epoch 16/50
6/6 [==============================] - 0s 11ms/step - loss: 1.1519 - accuracy: 0.6069 - val_loss: 1.4065 - val_accuracy: 0.3788
Epoch 17/50
6/6 [==============================] - 0s 15ms/step - loss: 1.1101 - accuracy: 0.6450 - val_loss: 1.3950 - val_accuracy: 0.4091
Epoch 18/50
6/6 [==============================] - 0s 11ms/step - loss: 1.0974 - accuracy: 0.6107 - val_loss: 1.3881 - val_accuracy: 0.3939
Epoch 19/50
6/6 [==============================] - 0s 11ms/step - loss: 1.0414 - accuracy: 0.6221 - val_loss: 1.3802 - val_accuracy: 0.4091
Epoch 20/50
6/6 [==============================] - 0s 12ms/step - loss: 1.0127 - accuracy: 0.6412 - val_loss: 1.3670 - val_accuracy: 0.4242
Epoch 21/50
6/6 [==============================] - 0s 12ms/step - loss: 0.9945 - accuracy: 0.6679 - val_loss: 1.3617 - val_accuracy: 0.3939
Epoch 22/50
6/6 [==============================] - 0s 13ms/step - loss: 0.9584 - accuracy: 0.6832 - val_loss: 1.3681 - val_accuracy: 0.4091
Epoch 23/50
6/6 [==============================] - 0s 13ms/step - loss: 0.9220 - accuracy: 0.7176 - val_loss: 1.3750 - val_accuracy: 0.3788
Epoch 24/50
6/6 [==============================] - 0s 12ms/step - loss: 0.8947 - accuracy: 0.7137 - val_loss: 1.3740 - val_accuracy: 0.4091
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.695122	0.385542	0.670088	0.339056

NN_Model_Tuned_TFIDFFull(X_train_tfidffull_smote, X_test_tfidffull, y_train_tfidffull_smote, y_test_tfidffull)

Model: "sequential_732"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_2202 (Dense)          (None, 50)                11000     
                                                                 
 dropout_11 (Dropout)        (None, 50)                0         
                                                                 
 dense_2203 (Dense)          (None, 50)                2550      
                                                                 
 dense_2204 (Dense)          (None, 5)                 255       
                                                                 
=================================================================
Total params: 13,805
Trainable params: 13,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
9/9 [==============================] - 1s 39ms/step - loss: 1.5858 - accuracy: 0.2409 - val_loss: 1.7876 - val_accuracy: 0.0000e+00
Epoch 2/50
9/9 [==============================] - 0s 8ms/step - loss: 1.5396 - accuracy: 0.2886 - val_loss: 1.9210 - val_accuracy: 0.0000e+00
Epoch 3/50
9/9 [==============================] - 0s 9ms/step - loss: 1.4896 - accuracy: 0.3659 - val_loss: 2.0438 - val_accuracy: 0.0000e+00
Epoch 4/50
9/9 [==============================] - 0s 8ms/step - loss: 1.4452 - accuracy: 0.4477 - val_loss: 2.1461 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.4	0.39759	0.330566	0.314944

Observations-

i. Smote datasets are better fit. Test accuracy is best around 40%. Best test F1 score is around 31% in TFIDF technique for full dataset.

Apply ANN model on Word2Vec-

Tuned_ANN(X_wv_df, y_wv_df)

4/4 [==============================] - 1s 4ms/step - loss: 1.6067 - accuracy: 0.2547
1/1 [==============================] - 0s 136ms/step - loss: 1.6015 - accuracy: 0.2619
4/4 [==============================] - 0s 4ms/step - loss: 1.6094 - accuracy: 0.1811
1/1 [==============================] - 0s 137ms/step - loss: 1.6049 - accuracy: 0.3415
4/4 [==============================] - 0s 3ms/step - loss: 1.6077 - accuracy: 0.2514
1/1 [==============================] - 0s 127ms/step - loss: 1.6015 - accuracy: 0.3171
4/4 [==============================] - 0s 3ms/step - loss: 1.6052 - accuracy: 0.2838
1/1 [==============================] - 0s 144ms/step - loss: 1.6085 - accuracy: 0.1951
4/4 [==============================] - 0s 3ms/step - loss: 1.6058 - accuracy: 0.2432
1/1 [==============================] - 0s 131ms/step - loss: 1.5953 - accuracy: 0.3171
4/4 [==============================] - 0s 4ms/step - loss: 1.6072 - accuracy: 0.2378
1/1 [==============================] - 0s 135ms/step - loss: 1.6007 - accuracy: 0.4390
4/4 [==============================] - 0s 4ms/step - loss: 1.6080 - accuracy: 0.2865
1/1 [==============================] - 0s 142ms/step - loss: 1.6029 - accuracy: 0.3171
4/4 [==============================] - 0s 3ms/step - loss: 1.6073 - accuracy: 0.1892
1/1 [==============================] - 0s 137ms/step - loss: 1.6054 - accuracy: 0.3171
4/4 [==============================] - 0s 3ms/step - loss: 1.6034 - accuracy: 0.2784
1/1 [==============================] - 0s 127ms/step - loss: 1.6088 - accuracy: 0.3171
4/4 [==============================] - 1s 3ms/step - loss: 1.6038 - accuracy: 0.3000
1/1 [==============================] - 0s 155ms/step - loss: 1.5957 - accuracy: 0.3659
4/4 [==============================] - 0s 4ms/step - loss: 1.6046 - accuracy: 0.2629
1/1 [==============================] - 0s 145ms/step - loss: 1.5944 - accuracy: 0.3571
4/4 [==============================] - 0s 4ms/step - loss: 1.6026 - accuracy: 0.2703
1/1 [==============================] - 0s 127ms/step - loss: 1.5906 - accuracy: 0.3415
4/4 [==============================] - 0s 7ms/step - loss: 1.6047 - accuracy: 0.2757
1/1 [==============================] - 0s 200ms/step - loss: 1.5927 - accuracy: 0.3902
4/4 [==============================] - 1s 5ms/step - loss: 1.6045 - accuracy: 0.2649
1/1 [==============================] - 0s 196ms/step - loss: 1.6059 - accuracy: 0.2683
4/4 [==============================] - 1s 6ms/step - loss: 1.6050 - accuracy: 0.3081
1/1 [==============================] - 0s 188ms/step - loss: 1.5924 - accuracy: 0.3415
4/4 [==============================] - 1s 4ms/step - loss: 1.6069 - accuracy: 0.2000
1/1 [==============================] - 0s 136ms/step - loss: 1.5967 - accuracy: 0.4146
4/4 [==============================] - 0s 4ms/step - loss: 1.6094 - accuracy: 0.1459
1/1 [==============================] - 0s 147ms/step - loss: 1.6015 - accuracy: 0.3171
4/4 [==============================] - 0s 4ms/step - loss: 1.6042 - accuracy: 0.3027
1/1 [==============================] - 0s 135ms/step - loss: 1.6007 - accuracy: 0.2927
4/4 [==============================] - 0s 4ms/step - loss: 1.6033 - accuracy: 0.3405
1/1 [==============================] - 0s 135ms/step - loss: 1.6078 - accuracy: 0.2927
4/4 [==============================] - 0s 3ms/step - loss: 1.6054 - accuracy: 0.2189
1/1 [==============================] - 0s 142ms/step - loss: 1.5941 - accuracy: 0.5122
4/4 [==============================] - 0s 4ms/step - loss: 1.6007 - accuracy: 0.3333
1/1 [==============================] - 0s 136ms/step - loss: 1.5839 - accuracy: 0.3571
4/4 [==============================] - 0s 4ms/step - loss: 1.6023 - accuracy: 0.2892
1/1 [==============================] - 0s 140ms/step - loss: 1.5878 - accuracy: 0.3415
4/4 [==============================] - 0s 4ms/step - loss: 1.6027 - accuracy: 0.3378
1/1 [==============================] - 0s 143ms/step - loss: 1.5855 - accuracy: 0.3902
4/4 [==============================] - 0s 4ms/step - loss: 1.6009 - accuracy: 0.3324
1/1 [==============================] - 0s 138ms/step - loss: 1.6064 - accuracy: 0.2195
4/4 [==============================] - 0s 4ms/step - loss: 1.6022 - accuracy: 0.3351
1/1 [==============================] - 0s 138ms/step - loss: 1.5827 - accuracy: 0.3415
4/4 [==============================] - 0s 5ms/step - loss: 1.6012 - accuracy: 0.3243
1/1 [==============================] - 0s 144ms/step - loss: 1.5818 - accuracy: 0.4390
4/4 [==============================] - 0s 4ms/step - loss: 1.6014 - accuracy: 0.3405
1/1 [==============================] - 0s 147ms/step - loss: 1.5866 - accuracy: 0.3171
4/4 [==============================] - 0s 5ms/step - loss: 1.6038 - accuracy: 0.2405
1/1 [==============================] - 0s 138ms/step - loss: 1.5988 - accuracy: 0.2927
4/4 [==============================] - 1s 8ms/step - loss: 1.6032 - accuracy: 0.2973
1/1 [==============================] - 0s 234ms/step - loss: 1.6073 - accuracy: 0.2927
4/4 [==============================] - 1s 7ms/step - loss: 1.6011 - accuracy: 0.2135
1/1 [==============================] - 0s 254ms/step - loss: 1.5844 - accuracy: 0.2683
4/4 [==============================] - 1s 8ms/step - loss: 1.6019 - accuracy: 0.2791
1/1 [==============================] - 0s 241ms/step - loss: 1.5827 - accuracy: 0.3571
4/4 [==============================] - 1s 9ms/step - loss: 1.5993 - accuracy: 0.2919
1/1 [==============================] - 0s 258ms/step - loss: 1.5820 - accuracy: 0.3415
4/4 [==============================] - 1s 6ms/step - loss: 1.5998 - accuracy: 0.2919
1/1 [==============================] - 0s 132ms/step - loss: 1.5735 - accuracy: 0.3902
4/4 [==============================] - 0s 7ms/step - loss: 1.5987 - accuracy: 0.3054
1/1 [==============================] - 0s 145ms/step - loss: 1.6038 - accuracy: 0.2195
4/4 [==============================] - 0s 4ms/step - loss: 1.6008 - accuracy: 0.3351
1/1 [==============================] - 0s 139ms/step - loss: 1.5785 - accuracy: 0.3415
4/4 [==============================] - 0s 6ms/step - loss: 1.5965 - accuracy: 0.3243
1/1 [==============================] - 0s 137ms/step - loss: 1.5665 - accuracy: 0.4390
4/4 [==============================] - 0s 5ms/step - loss: 1.5977 - accuracy: 0.3378
1/1 [==============================] - 0s 126ms/step - loss: 1.5732 - accuracy: 0.3171
4/4 [==============================] - 0s 5ms/step - loss: 1.5998 - accuracy: 0.2514
1/1 [==============================] - 0s 135ms/step - loss: 1.5925 - accuracy: 0.2927
4/4 [==============================] - 0s 5ms/step - loss: 1.5967 - accuracy: 0.3405
1/1 [==============================] - 0s 131ms/step - loss: 1.6050 - accuracy: 0.2927
4/4 [==============================] - 0s 5ms/step - loss: 1.5992 - accuracy: 0.3243
1/1 [==============================] - 0s 124ms/step - loss: 1.5818 - accuracy: 0.3659
4/4 [==============================] - 0s 6ms/step - loss: 1.6051 - accuracy: 0.1870
1/1 [==============================] - 0s 136ms/step - loss: 1.5960 - accuracy: 0.2381
4/4 [==============================] - 0s 3ms/step - loss: 1.6070 - accuracy: 0.2405
1/1 [==============================] - 0s 137ms/step - loss: 1.5997 - accuracy: 0.3659
4/4 [==============================] - 1s 4ms/step - loss: 1.6071 - accuracy: 0.2919
1/1 [==============================] - 0s 132ms/step - loss: 1.5985 - accuracy: 0.3902
4/4 [==============================] - 0s 4ms/step - loss: 1.6067 - accuracy: 0.3027
1/1 [==============================] - 0s 133ms/step - loss: 1.6088 - accuracy: 0.2195
4/4 [==============================] - 0s 4ms/step - loss: 1.6053 - accuracy: 0.2919
1/1 [==============================] - 0s 203ms/step - loss: 1.5942 - accuracy: 0.3415
4/4 [==============================] - 1s 5ms/step - loss: 1.6089 - accuracy: 0.2378
1/1 [==============================] - 0s 206ms/step - loss: 1.6017 - accuracy: 0.4390
4/4 [==============================] - 1s 5ms/step - loss: 1.6056 - accuracy: 0.2649
1/1 [==============================] - 0s 245ms/step - loss: 1.5967 - accuracy: 0.3171
4/4 [==============================] - 1s 4ms/step - loss: 1.6079 - accuracy: 0.2135
1/1 [==============================] - 0s 148ms/step - loss: 1.6052 - accuracy: 0.4146
4/4 [==============================] - 0s 4ms/step - loss: 1.6067 - accuracy: 0.2243
1/1 [==============================] - 0s 143ms/step - loss: 1.6077 - accuracy: 0.2927
4/4 [==============================] - 0s 3ms/step - loss: 1.6037 - accuracy: 0.3243
1/1 [==============================] - 0s 141ms/step - loss: 1.5979 - accuracy: 0.3659
4/4 [==============================] - 0s 5ms/step - loss: 1.6060 - accuracy: 0.2602
1/1 [==============================] - 0s 135ms/step - loss: 1.5978 - accuracy: 0.3571
4/4 [==============================] - 0s 4ms/step - loss: 1.6028 - accuracy: 0.2892
1/1 [==============================] - 0s 133ms/step - loss: 1.5968 - accuracy: 0.3415
4/4 [==============================] - 0s 4ms/step - loss: 1.6040 - accuracy: 0.2757
1/1 [==============================] - 0s 160ms/step - loss: 1.5897 - accuracy: 0.3902
4/4 [==============================] - 0s 4ms/step - loss: 1.6038 - accuracy: 0.3486
1/1 [==============================] - 0s 139ms/step - loss: 1.6084 - accuracy: 0.2195
4/4 [==============================] - 0s 4ms/step - loss: 1.6025 - accuracy: 0.3027
1/1 [==============================] - 0s 146ms/step - loss: 1.5874 - accuracy: 0.3415
4/4 [==============================] - 0s 4ms/step - loss: 1.6046 - accuracy: 0.2270
1/1 [==============================] - 0s 128ms/step - loss: 1.5932 - accuracy: 0.2683
4/4 [==============================] - 0s 4ms/step - loss: 1.6051 - accuracy: 0.1919
1/1 [==============================] - 0s 137ms/step - loss: 1.5927 - accuracy: 0.3171
4/4 [==============================] - 0s 4ms/step - loss: 1.6048 - accuracy: 0.3000
1/1 [==============================] - 0s 138ms/step - loss: 1.6026 - accuracy: 0.2927
4/4 [==============================] - 0s 5ms/step - loss: 1.6021 - accuracy: 0.2676
1/1 [==============================] - 0s 158ms/step - loss: 1.6079 - accuracy: 0.1707
4/4 [==============================] - 0s 4ms/step - loss: 1.6055 - accuracy: 0.3270
1/1 [==============================] - 0s 138ms/step - loss: 1.5912 - accuracy: 0.3659
4/4 [==============================] - 0s 5ms/step - loss: 1.6022 - accuracy: 0.2873
1/1 [==============================] - 0s 149ms/step - loss: 1.5850 - accuracy: 0.3571
4/4 [==============================] - 1s 6ms/step - loss: 1.6019 - accuracy: 0.2892
1/1 [==============================] - 0s 206ms/step - loss: 1.5887 - accuracy: 0.3415
4/4 [==============================] - 1s 10ms/step - loss: 1.6009 - accuracy: 0.3297
1/1 [==============================] - 0s 228ms/step - loss: 1.5813 - accuracy: 0.3902
4/4 [==============================] - 1s 9ms/step - loss: 1.6026 - accuracy: 0.2946
1/1 [==============================] - 0s 150ms/step - loss: 1.6057 - accuracy: 0.2195
4/4 [==============================] - 0s 5ms/step - loss: 1.5987 - accuracy: 0.3351
1/1 [==============================] - 0s 145ms/step - loss: 1.5759 - accuracy: 0.3415
4/4 [==============================] - 0s 5ms/step - loss: 1.5997 - accuracy: 0.2811
1/1 [==============================] - 0s 143ms/step - loss: 1.5779 - accuracy: 0.2195
4/4 [==============================] - 0s 5ms/step - loss: 1.6053 - accuracy: 0.1838
1/1 [==============================] - 0s 145ms/step - loss: 1.5907 - accuracy: 0.3171
4/4 [==============================] - 0s 5ms/step - loss: 1.6034 - accuracy: 0.2405
1/1 [==============================] - 0s 138ms/step - loss: 1.5992 - accuracy: 0.1463
4/4 [==============================] - 0s 5ms/step - loss: 1.5978 - accuracy: 0.3432
1/1 [==============================] - 0s 127ms/step - loss: 1.6081 - accuracy: 0.2927
4/4 [==============================] - 0s 5ms/step - loss: 1.6011 - accuracy: 0.2622
1/1 [==============================] - 0s 145ms/step - loss: 1.5883 - accuracy: 0.3659
4/4 [==============================] - 0s 5ms/step - loss: 1.6025 - accuracy: 0.2683
1/1 [==============================] - 0s 130ms/step - loss: 1.5864 - accuracy: 0.3571
4/4 [==============================] - 0s 5ms/step - loss: 1.5995 - accuracy: 0.3135
1/1 [==============================] - 0s 138ms/step - loss: 1.5829 - accuracy: 0.3415
4/4 [==============================] - 0s 5ms/step - loss: 1.5998 - accuracy: 0.3108
1/1 [==============================] - 0s 141ms/step - loss: 1.5756 - accuracy: 0.3902
4/4 [==============================] - 0s 6ms/step - loss: 1.5995 - accuracy: 0.3162
1/1 [==============================] - 0s 136ms/step - loss: 1.6079 - accuracy: 0.2195
4/4 [==============================] - 0s 5ms/step - loss: 1.6001 - accuracy: 0.3000
1/1 [==============================] - 0s 139ms/step - loss: 1.5751 - accuracy: 0.3415
4/4 [==============================] - 0s 5ms/step - loss: 1.5986 - accuracy: 0.3243
1/1 [==============================] - 0s 202ms/step - loss: 1.5688 - accuracy: 0.4390
4/4 [==============================] - 1s 7ms/step - loss: 1.6000 - accuracy: 0.2568
1/1 [==============================] - 0s 195ms/step - loss: 1.5778 - accuracy: 0.3171
4/4 [==============================] - 1s 9ms/step - loss: 1.6018 - accuracy: 0.2108
1/1 [==============================] - 0s 197ms/step - loss: 1.5943 - accuracy: 0.2927
4/4 [==============================] - 1s 5ms/step - loss: 1.5982 - accuracy: 0.3324
1/1 [==============================] - 0s 132ms/step - loss: 1.6067 - accuracy: 0.2927
4/4 [==============================] - 0s 7ms/step - loss: 1.6005 - accuracy: 0.2703
1/1 [==============================] - 0s 135ms/step - loss: 1.5874 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.5987 - accuracy: 0.2629
3/3 [==============================] - 0s 4ms/step - loss: 1.5772 - accuracy: 0.3571
19/19 [==============================] - 0s 2ms/step - loss: 1.5994 - accuracy: 0.3027
3/3 [==============================] - 0s 4ms/step - loss: 1.5856 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5987 - accuracy: 0.3108
3/3 [==============================] - 0s 3ms/step - loss: 1.5731 - accuracy: 0.3902
19/19 [==============================] - 0s 2ms/step - loss: 1.5919 - accuracy: 0.3189
3/3 [==============================] - 0s 2ms/step - loss: 1.6040 - accuracy: 0.2683
19/19 [==============================] - 0s 2ms/step - loss: 1.5942 - accuracy: 0.2162
3/3 [==============================] - 0s 3ms/step - loss: 1.5513 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.5909 - accuracy: 0.2622
3/3 [==============================] - 0s 3ms/step - loss: 1.5577 - accuracy: 0.1951
19/19 [==============================] - 0s 2ms/step - loss: 1.5967 - accuracy: 0.3189
3/3 [==============================] - 0s 3ms/step - loss: 1.5739 - accuracy: 0.3171
19/19 [==============================] - 0s 3ms/step - loss: 1.5835 - accuracy: 0.2432
3/3 [==============================] - 0s 3ms/step - loss: 1.5828 - accuracy: 0.1463
19/19 [==============================] - 0s 2ms/step - loss: 1.5899 - accuracy: 0.3378
3/3 [==============================] - 0s 3ms/step - loss: 1.6033 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5942 - accuracy: 0.3162
3/3 [==============================] - 0s 3ms/step - loss: 1.5809 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.5864 - accuracy: 0.2818
3/3 [==============================] - 0s 4ms/step - loss: 1.5466 - accuracy: 0.3571
19/19 [==============================] - 1s 4ms/step - loss: 1.5749 - accuracy: 0.3270
3/3 [==============================] - 0s 5ms/step - loss: 1.5296 - accuracy: 0.3415
19/19 [==============================] - 1s 4ms/step - loss: 1.5816 - accuracy: 0.2135
3/3 [==============================] - 0s 3ms/step - loss: 1.5205 - accuracy: 0.3902
19/19 [==============================] - 1s 4ms/step - loss: 1.5864 - accuracy: 0.3324
3/3 [==============================] - 0s 6ms/step - loss: 1.6100 - accuracy: 0.2195
19/19 [==============================] - 0s 2ms/step - loss: 1.5867 - accuracy: 0.3216
3/3 [==============================] - 0s 3ms/step - loss: 1.5356 - accuracy: 0.3415
19/19 [==============================] - 0s 3ms/step - loss: 1.5827 - accuracy: 0.3054
3/3 [==============================] - 0s 3ms/step - loss: 1.5186 - accuracy: 0.4390
19/19 [==============================] - 0s 2ms/step - loss: 1.5896 - accuracy: 0.3189
3/3 [==============================] - 0s 3ms/step - loss: 1.5473 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.5794 - accuracy: 0.2811
3/3 [==============================] - 0s 3ms/step - loss: 1.5660 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5919 - accuracy: 0.2378
3/3 [==============================] - 0s 3ms/step - loss: 1.6183 - accuracy: 0.1220
19/19 [==============================] - 0s 2ms/step - loss: 1.5883 - accuracy: 0.3027
3/3 [==============================] - 0s 3ms/step - loss: 1.5625 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.5784 - accuracy: 0.2818
3/3 [==============================] - 0s 3ms/step - loss: 1.5211 - accuracy: 0.3571
19/19 [==============================] - 0s 2ms/step - loss: 1.5676 - accuracy: 0.3351
3/3 [==============================] - 0s 3ms/step - loss: 1.5205 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5664 - accuracy: 0.3162
3/3 [==============================] - 0s 3ms/step - loss: 1.4616 - accuracy: 0.3902
19/19 [==============================] - 0s 2ms/step - loss: 1.5569 - accuracy: 0.3378
3/3 [==============================] - 0s 4ms/step - loss: 1.6180 - accuracy: 0.2195
19/19 [==============================] - 0s 2ms/step - loss: 1.5646 - accuracy: 0.3351
3/3 [==============================] - 0s 3ms/step - loss: 1.4614 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5683 - accuracy: 0.3135
3/3 [==============================] - 0s 3ms/step - loss: 1.4767 - accuracy: 0.4390
19/19 [==============================] - 0s 2ms/step - loss: 1.5763 - accuracy: 0.3270
3/3 [==============================] - 0s 4ms/step - loss: 1.5023 - accuracy: 0.3171
19/19 [==============================] - 1s 4ms/step - loss: 1.5829 - accuracy: 0.2324
3/3 [==============================] - 1s 3ms/step - loss: 1.5673 - accuracy: 0.2927
19/19 [==============================] - 1s 2ms/step - loss: 1.5659 - accuracy: 0.2892
3/3 [==============================] - 0s 3ms/step - loss: 1.6258 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5602 - accuracy: 0.3243
3/3 [==============================] - 0s 3ms/step - loss: 1.4889 - accuracy: 0.3659
19/19 [==============================] - 0s 3ms/step - loss: 1.5629 - accuracy: 0.2900
3/3 [==============================] - 0s 4ms/step - loss: 1.4773 - accuracy: 0.3571
19/19 [==============================] - 0s 3ms/step - loss: 1.5593 - accuracy: 0.3135
3/3 [==============================] - 0s 3ms/step - loss: 1.4904 - accuracy: 0.3415
19/19 [==============================] - 0s 3ms/step - loss: 1.5513 - accuracy: 0.3162
3/3 [==============================] - 0s 3ms/step - loss: 1.4023 - accuracy: 0.3902
19/19 [==============================] - 0s 3ms/step - loss: 1.5276 - accuracy: 0.3324
3/3 [==============================] - 0s 3ms/step - loss: 1.6717 - accuracy: 0.2195
19/19 [==============================] - 0s 3ms/step - loss: 1.5507 - accuracy: 0.3216
3/3 [==============================] - 0s 4ms/step - loss: 1.4311 - accuracy: 0.3415
19/19 [==============================] - 0s 3ms/step - loss: 1.5411 - accuracy: 0.3243
3/3 [==============================] - 0s 3ms/step - loss: 1.3881 - accuracy: 0.4390
19/19 [==============================] - 1s 8ms/step - loss: 1.5587 - accuracy: 0.3081
3/3 [==============================] - 0s 4ms/step - loss: 1.4507 - accuracy: 0.3171
19/19 [==============================] - 1s 6ms/step - loss: 1.5507 - accuracy: 0.2784
3/3 [==============================] - 0s 5ms/step - loss: 1.5370 - accuracy: 0.2927
19/19 [==============================] - 1s 5ms/step - loss: 1.5418 - accuracy: 0.2838
3/3 [==============================] - 0s 9ms/step - loss: 1.6705 - accuracy: 0.2927
19/19 [==============================] - 2s 6ms/step - loss: 1.5587 - accuracy: 0.3189
3/3 [==============================] - 0s 5ms/step - loss: 1.4924 - accuracy: 0.3659
19/19 [==============================] - 1s 5ms/step - loss: 1.5936 - accuracy: 0.3089
3/3 [==============================] - 0s 8ms/step - loss: 1.5663 - accuracy: 0.3571
19/19 [==============================] - 2s 4ms/step - loss: 1.5975 - accuracy: 0.2378
3/3 [==============================] - 0s 6ms/step - loss: 1.5876 - accuracy: 0.1707
19/19 [==============================] - 1s 5ms/step - loss: 1.5963 - accuracy: 0.3189
3/3 [==============================] - 0s 6ms/step - loss: 1.5667 - accuracy: 0.3902
19/19 [==============================] - 1s 5ms/step - loss: 1.5959 - accuracy: 0.3324
3/3 [==============================] - 0s 5ms/step - loss: 1.6032 - accuracy: 0.2195
19/19 [==============================] - 1s 4ms/step - loss: 1.5915 - accuracy: 0.3189
3/3 [==============================] - 0s 4ms/step - loss: 1.5480 - accuracy: 0.3415
19/19 [==============================] - 1s 7ms/step - loss: 1.6028 - accuracy: 0.2054
3/3 [==============================] - 0s 5ms/step - loss: 1.5858 - accuracy: 0.4390
19/19 [==============================] - 1s 4ms/step - loss: 1.5952 - accuracy: 0.3243
3/3 [==============================] - 0s 7ms/step - loss: 1.5684 - accuracy: 0.3171
19/19 [==============================] - 1s 4ms/step - loss: 1.5808 - accuracy: 0.2432
3/3 [==============================] - 0s 5ms/step - loss: 1.5809 - accuracy: 0.2927
19/19 [==============================] - 1s 5ms/step - loss: 1.5917 - accuracy: 0.2432
3/3 [==============================] - 0s 9ms/step - loss: 1.6095 - accuracy: 0.2927
19/19 [==============================] - 2s 4ms/step - loss: 1.5901 - accuracy: 0.3324
3/3 [==============================] - 0s 5ms/step - loss: 1.5724 - accuracy: 0.3659
19/19 [==============================] - 1s 5ms/step - loss: 1.5967 - accuracy: 0.2954
3/3 [==============================] - 0s 7ms/step - loss: 1.5716 - accuracy: 0.3571
19/19 [==============================] - 1s 5ms/step - loss: 1.5955 - accuracy: 0.2946
3/3 [==============================] - 0s 6ms/step - loss: 1.5652 - accuracy: 0.4146
19/19 [==============================] - 1s 5ms/step - loss: 1.5863 - accuracy: 0.3081
3/3 [==============================] - 0s 4ms/step - loss: 1.5321 - accuracy: 0.3902
19/19 [==============================] - 1s 4ms/step - loss: 1.5826 - accuracy: 0.3351
3/3 [==============================] - 0s 6ms/step - loss: 1.5925 - accuracy: 0.2195
19/19 [==============================] - 1s 4ms/step - loss: 1.5826 - accuracy: 0.3189
3/3 [==============================] - 0s 4ms/step - loss: 1.5152 - accuracy: 0.3415
19/19 [==============================] - 1s 3ms/step - loss: 1.5854 - accuracy: 0.2270
3/3 [==============================] - 0s 5ms/step - loss: 1.5287 - accuracy: 0.4390
19/19 [==============================] - 1s 4ms/step - loss: 1.5874 - accuracy: 0.3459
3/3 [==============================] - 0s 6ms/step - loss: 1.5415 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.5793 - accuracy: 0.3324
3/3 [==============================] - 0s 3ms/step - loss: 1.5716 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5808 - accuracy: 0.2405
3/3 [==============================] - 0s 3ms/step - loss: 1.6211 - accuracy: 0.1220
19/19 [==============================] - 0s 2ms/step - loss: 1.5800 - accuracy: 0.3270
3/3 [==============================] - 0s 3ms/step - loss: 1.5356 - accuracy: 0.3659
19/19 [==============================] - 1s 2ms/step - loss: 1.5750 - accuracy: 0.2737
3/3 [==============================] - 0s 3ms/step - loss: 1.5107 - accuracy: 0.3571
19/19 [==============================] - 0s 2ms/step - loss: 1.5779 - accuracy: 0.3162
3/3 [==============================] - 0s 4ms/step - loss: 1.5353 - accuracy: 0.3415
19/19 [==============================] - 0s 3ms/step - loss: 1.5737 - accuracy: 0.3135
3/3 [==============================] - 0s 5ms/step - loss: 1.4834 - accuracy: 0.3902
19/19 [==============================] - 1s 4ms/step - loss: 1.5596 - accuracy: 0.3486
3/3 [==============================] - 0s 4ms/step - loss: 1.6222 - accuracy: 0.2195
19/19 [==============================] - 1s 3ms/step - loss: 1.5671 - accuracy: 0.3351
3/3 [==============================] - 0s 3ms/step - loss: 1.4644 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5783 - accuracy: 0.2595
3/3 [==============================] - 0s 3ms/step - loss: 1.4989 - accuracy: 0.4390
19/19 [==============================] - 0s 2ms/step - loss: 1.5765 - accuracy: 0.3297
3/3 [==============================] - 0s 3ms/step - loss: 1.5041 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.5755 - accuracy: 0.2676
3/3 [==============================] - 0s 7ms/step - loss: 1.5601 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5629 - accuracy: 0.2892
3/3 [==============================] - 0s 4ms/step - loss: 1.6297 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5724 - accuracy: 0.3054
3/3 [==============================] - 0s 4ms/step - loss: 1.5256 - accuracy: 0.3659
19/19 [==============================] - 0s 3ms/step - loss: 1.5615 - accuracy: 0.2927
3/3 [==============================] - 0s 4ms/step - loss: 1.4773 - accuracy: 0.3571
19/19 [==============================] - 1s 3ms/step - loss: 1.5447 - accuracy: 0.3189
3/3 [==============================] - 0s 4ms/step - loss: 1.4548 - accuracy: 0.3415
19/19 [==============================] - 1s 3ms/step - loss: 1.5512 - accuracy: 0.3162
3/3 [==============================] - 0s 3ms/step - loss: 1.4100 - accuracy: 0.3902
19/19 [==============================] - 1s 4ms/step - loss: 1.5373 - accuracy: 0.3378
3/3 [==============================] - 0s 4ms/step - loss: 1.6433 - accuracy: 0.2195
19/19 [==============================] - 0s 3ms/step - loss: 1.5452 - accuracy: 0.3378
3/3 [==============================] - 0s 4ms/step - loss: 1.3929 - accuracy: 0.3415
19/19 [==============================] - 0s 3ms/step - loss: 1.5504 - accuracy: 0.3243
3/3 [==============================] - 0s 4ms/step - loss: 1.4093 - accuracy: 0.4390
19/19 [==============================] - 0s 3ms/step - loss: 1.5612 - accuracy: 0.3378
3/3 [==============================] - 0s 4ms/step - loss: 1.4600 - accuracy: 0.3171
19/19 [==============================] - 1s 5ms/step - loss: 1.5595 - accuracy: 0.3270
3/3 [==============================] - 0s 5ms/step - loss: 1.5413 - accuracy: 0.2927
19/19 [==============================] - 1s 4ms/step - loss: 1.5481 - accuracy: 0.2919
3/3 [==============================] - 0s 3ms/step - loss: 1.6627 - accuracy: 0.2927
19/19 [==============================] - 1s 5ms/step - loss: 1.5661 - accuracy: 0.3162
3/3 [==============================] - 0s 3ms/step - loss: 1.5039 - accuracy: 0.3659
8/8 [==============================] - 0s 3ms/step - loss: 1.5986 - accuracy: 0.3333
1/1 [==============================] - 0s 139ms/step - loss: 1.5838 - accuracy: 0.3571
8/8 [==============================] - 0s 2ms/step - loss: 1.6035 - accuracy: 0.2757
1/1 [==============================] - 0s 129ms/step - loss: 1.5932 - accuracy: 0.3415
8/8 [==============================] - 0s 2ms/step - loss: 1.5957 - accuracy: 0.3162
1/1 [==============================] - 0s 128ms/step - loss: 1.5700 - accuracy: 0.3902
8/8 [==============================] - 0s 2ms/step - loss: 1.6065 - accuracy: 0.2216
1/1 [==============================] - 0s 129ms/step - loss: 1.6120 - accuracy: 0.1220
8/8 [==============================] - 0s 3ms/step - loss: 1.6024 - accuracy: 0.3108
1/1 [==============================] - 0s 143ms/step - loss: 1.5857 - accuracy: 0.3415
8/8 [==============================] - 0s 2ms/step - loss: 1.6043 - accuracy: 0.2514
1/1 [==============================] - 0s 133ms/step - loss: 1.5918 - accuracy: 0.4390
8/8 [==============================] - 0s 2ms/step - loss: 1.6062 - accuracy: 0.3081
1/1 [==============================] - 0s 135ms/step - loss: 1.5988 - accuracy: 0.3171
8/8 [==============================] - 0s 2ms/step - loss: 1.6038 - accuracy: 0.2405
1/1 [==============================] - 0s 131ms/step - loss: 1.6047 - accuracy: 0.1463
8/8 [==============================] - 0s 3ms/step - loss: 1.6019 - accuracy: 0.3081
1/1 [==============================] - 0s 138ms/step - loss: 1.6082 - accuracy: 0.2927
8/8 [==============================] - 0s 2ms/step - loss: 1.5990 - accuracy: 0.2676
1/1 [==============================] - 0s 146ms/step - loss: 1.5914 - accuracy: 0.1463
8/8 [==============================] - 0s 3ms/step - loss: 1.6013 - accuracy: 0.2954
1/1 [==============================] - 0s 130ms/step - loss: 1.5864 - accuracy: 0.3571
8/8 [==============================] - 0s 3ms/step - loss: 1.5995 - accuracy: 0.3027
1/1 [==============================] - 0s 129ms/step - loss: 1.5848 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5992 - accuracy: 0.3135
1/1 [==============================] - 0s 149ms/step - loss: 1.5749 - accuracy: 0.3902
8/8 [==============================] - 1s 4ms/step - loss: 1.5988 - accuracy: 0.3324
1/1 [==============================] - 0s 183ms/step - loss: 1.6106 - accuracy: 0.2195
8/8 [==============================] - 2s 4ms/step - loss: 1.5968 - accuracy: 0.3135
1/1 [==============================] - 0s 135ms/step - loss: 1.5662 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5994 - accuracy: 0.3243
1/1 [==============================] - 0s 163ms/step - loss: 1.5746 - accuracy: 0.4390
8/8 [==============================] - 0s 3ms/step - loss: 1.6000 - accuracy: 0.3378
1/1 [==============================] - 0s 142ms/step - loss: 1.5826 - accuracy: 0.3171
8/8 [==============================] - 0s 3ms/step - loss: 1.6004 - accuracy: 0.2000
1/1 [==============================] - 0s 142ms/step - loss: 1.5979 - accuracy: 0.1463
8/8 [==============================] - 0s 3ms/step - loss: 1.5935 - accuracy: 0.2649
1/1 [==============================] - 0s 133ms/step - loss: 1.6112 - accuracy: 0.1707
8/8 [==============================] - 0s 3ms/step - loss: 1.5997 - accuracy: 0.3000
1/1 [==============================] - 0s 144ms/step - loss: 1.5853 - accuracy: 0.3659
8/8 [==============================] - 0s 3ms/step - loss: 1.5937 - accuracy: 0.2737
1/1 [==============================] - 0s 133ms/step - loss: 1.5653 - accuracy: 0.3571
8/8 [==============================] - 0s 3ms/step - loss: 1.5985 - accuracy: 0.2649
1/1 [==============================] - 0s 146ms/step - loss: 1.5755 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5929 - accuracy: 0.3027
1/1 [==============================] - 0s 135ms/step - loss: 1.5538 - accuracy: 0.3902
8/8 [==============================] - 0s 3ms/step - loss: 1.5895 - accuracy: 0.3054
1/1 [==============================] - 0s 136ms/step - loss: 1.6010 - accuracy: 0.2195
8/8 [==============================] - 0s 3ms/step - loss: 1.5905 - accuracy: 0.3432
1/1 [==============================] - 0s 147ms/step - loss: 1.5536 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5937 - accuracy: 0.2378
1/1 [==============================] - 0s 138ms/step - loss: 1.5553 - accuracy: 0.4390
8/8 [==============================] - 0s 3ms/step - loss: 1.5955 - accuracy: 0.2270
1/1 [==============================] - 0s 140ms/step - loss: 1.5640 - accuracy: 0.3171
8/8 [==============================] - 0s 4ms/step - loss: 1.5931 - accuracy: 0.3054
1/1 [==============================] - 0s 193ms/step - loss: 1.5847 - accuracy: 0.2927
8/8 [==============================] - 1s 5ms/step - loss: 1.5906 - accuracy: 0.2595
1/1 [==============================] - 0s 203ms/step - loss: 1.6113 - accuracy: 0.2927
8/8 [==============================] - 1s 5ms/step - loss: 1.6003 - accuracy: 0.3081
1/1 [==============================] - 0s 217ms/step - loss: 1.5875 - accuracy: 0.3659
8/8 [==============================] - 1s 4ms/step - loss: 1.5966 - accuracy: 0.2656
1/1 [==============================] - 0s 125ms/step - loss: 1.5670 - accuracy: 0.3571
8/8 [==============================] - 0s 4ms/step - loss: 1.5898 - accuracy: 0.3324
1/1 [==============================] - 0s 145ms/step - loss: 1.5574 - accuracy: 0.3415
8/8 [==============================] - 0s 4ms/step - loss: 1.5863 - accuracy: 0.3189
1/1 [==============================] - 0s 143ms/step - loss: 1.5293 - accuracy: 0.3902
8/8 [==============================] - 0s 4ms/step - loss: 1.5888 - accuracy: 0.3189
1/1 [==============================] - 0s 130ms/step - loss: 1.6067 - accuracy: 0.2195
8/8 [==============================] - 0s 5ms/step - loss: 1.5920 - accuracy: 0.3081
1/1 [==============================] - 0s 125ms/step - loss: 1.5358 - accuracy: 0.3415
8/8 [==============================] - 0s 4ms/step - loss: 1.5941 - accuracy: 0.2000
1/1 [==============================] - 0s 135ms/step - loss: 1.5527 - accuracy: 0.4390
8/8 [==============================] - 0s 4ms/step - loss: 1.5951 - accuracy: 0.1973
1/1 [==============================] - 0s 140ms/step - loss: 1.5532 - accuracy: 0.3171
8/8 [==============================] - 0s 4ms/step - loss: 1.5931 - accuracy: 0.2243
1/1 [==============================] - 0s 129ms/step - loss: 1.5836 - accuracy: 0.2927
8/8 [==============================] - 0s 4ms/step - loss: 1.5859 - accuracy: 0.3378
1/1 [==============================] - 0s 126ms/step - loss: 1.6086 - accuracy: 0.2927
8/8 [==============================] - 0s 4ms/step - loss: 1.5911 - accuracy: 0.2703
1/1 [==============================] - 0s 122ms/step - loss: 1.5694 - accuracy: 0.1463
8/8 [==============================] - 1s 4ms/step - loss: 1.6031 - accuracy: 0.3117
1/1 [==============================] - 0s 178ms/step - loss: 1.5919 - accuracy: 0.3571
8/8 [==============================] - 1s 3ms/step - loss: 1.5978 - accuracy: 0.2973
1/1 [==============================] - 0s 184ms/step - loss: 1.5842 - accuracy: 0.3415
8/8 [==============================] - 1s 3ms/step - loss: 1.6011 - accuracy: 0.2892
1/1 [==============================] - 0s 215ms/step - loss: 1.5841 - accuracy: 0.3902
8/8 [==============================] - 1s 4ms/step - loss: 1.6040 - accuracy: 0.3324
1/1 [==============================] - 0s 186ms/step - loss: 1.6102 - accuracy: 0.2195
8/8 [==============================] - 1s 4ms/step - loss: 1.6027 - accuracy: 0.3189
1/1 [==============================] - 0s 186ms/step - loss: 1.5865 - accuracy: 0.3415
8/8 [==============================] - 1s 3ms/step - loss: 1.6034 - accuracy: 0.2108
1/1 [==============================] - 0s 171ms/step - loss: 1.5885 - accuracy: 0.4146
8/8 [==============================] - 0s 3ms/step - loss: 1.6045 - accuracy: 0.3027
1/1 [==============================] - 0s 128ms/step - loss: 1.5935 - accuracy: 0.3171
8/8 [==============================] - 1s 2ms/step - loss: 1.6073 - accuracy: 0.2000
1/1 [==============================] - 0s 134ms/step - loss: 1.6029 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.6001 - accuracy: 0.3405
1/1 [==============================] - 0s 127ms/step - loss: 1.6049 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.5986 - accuracy: 0.3162
1/1 [==============================] - 0s 132ms/step - loss: 1.5877 - accuracy: 0.3659
8/8 [==============================] - 0s 3ms/step - loss: 1.6047 - accuracy: 0.2629
1/1 [==============================] - 0s 150ms/step - loss: 1.5932 - accuracy: 0.3571
8/8 [==============================] - 0s 3ms/step - loss: 1.5996 - accuracy: 0.3135
1/1 [==============================] - 0s 129ms/step - loss: 1.5888 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.6010 - accuracy: 0.2811
1/1 [==============================] - 0s 125ms/step - loss: 1.5802 - accuracy: 0.3902
8/8 [==============================] - 0s 3ms/step - loss: 1.5905 - accuracy: 0.3486
1/1 [==============================] - 0s 129ms/step - loss: 1.6049 - accuracy: 0.2195
8/8 [==============================] - 0s 3ms/step - loss: 1.5987 - accuracy: 0.3189
1/1 [==============================] - 0s 154ms/step - loss: 1.5696 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5992 - accuracy: 0.3243
1/1 [==============================] - 0s 134ms/step - loss: 1.5723 - accuracy: 0.4390
8/8 [==============================] - 0s 3ms/step - loss: 1.6007 - accuracy: 0.2649
1/1 [==============================] - 0s 129ms/step - loss: 1.5808 - accuracy: 0.3171
8/8 [==============================] - 0s 4ms/step - loss: 1.6032 - accuracy: 0.2000
1/1 [==============================] - 0s 126ms/step - loss: 1.5994 - accuracy: 0.2927
8/8 [==============================] - 1s 4ms/step - loss: 1.5987 - accuracy: 0.2486
1/1 [==============================] - 0s 222ms/step - loss: 1.6129 - accuracy: 0.1707
8/8 [==============================] - 1s 5ms/step - loss: 1.6010 - accuracy: 0.3297
1/1 [==============================] - 0s 218ms/step - loss: 1.5835 - accuracy: 0.3659
8/8 [==============================] - 1s 5ms/step - loss: 1.5944 - accuracy: 0.3035
1/1 [==============================] - 0s 212ms/step - loss: 1.5670 - accuracy: 0.3571
8/8 [==============================] - 0s 4ms/step - loss: 1.5959 - accuracy: 0.2865
1/1 [==============================] - 0s 153ms/step - loss: 1.5725 - accuracy: 0.3415
8/8 [==============================] - 0s 4ms/step - loss: 1.5986 - accuracy: 0.3189
1/1 [==============================] - 0s 160ms/step - loss: 1.5708 - accuracy: 0.3902
8/8 [==============================] - 0s 4ms/step - loss: 1.5900 - accuracy: 0.3324
1/1 [==============================] - 0s 135ms/step - loss: 1.6143 - accuracy: 0.2195
8/8 [==============================] - 0s 3ms/step - loss: 1.5883 - accuracy: 0.3351
1/1 [==============================] - 0s 130ms/step - loss: 1.5483 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5953 - accuracy: 0.3243
1/1 [==============================] - 0s 148ms/step - loss: 1.5531 - accuracy: 0.4390
8/8 [==============================] - 0s 3ms/step - loss: 1.5975 - accuracy: 0.2486
1/1 [==============================] - 0s 136ms/step - loss: 1.5684 - accuracy: 0.3171
8/8 [==============================] - 0s 3ms/step - loss: 1.5953 - accuracy: 0.2405
1/1 [==============================] - 0s 130ms/step - loss: 1.5879 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.5912 - accuracy: 0.2973
1/1 [==============================] - 0s 123ms/step - loss: 1.6073 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.5837 - accuracy: 0.3324
1/1 [==============================] - 0s 145ms/step - loss: 1.5496 - accuracy: 0.3659
8/8 [==============================] - 0s 4ms/step - loss: 1.5886 - accuracy: 0.2873
1/1 [==============================] - 0s 144ms/step - loss: 1.5455 - accuracy: 0.3571
8/8 [==============================] - 0s 4ms/step - loss: 1.5946 - accuracy: 0.3135
1/1 [==============================] - 0s 134ms/step - loss: 1.5682 - accuracy: 0.3415
8/8 [==============================] - 0s 4ms/step - loss: 1.5892 - accuracy: 0.2811
1/1 [==============================] - 0s 137ms/step - loss: 1.5370 - accuracy: 0.3902
8/8 [==============================] - 0s 4ms/step - loss: 1.5815 - accuracy: 0.3486
1/1 [==============================] - 0s 138ms/step - loss: 1.6085 - accuracy: 0.2195
8/8 [==============================] - 1s 6ms/step - loss: 1.5899 - accuracy: 0.3081
1/1 [==============================] - 0s 189ms/step - loss: 1.5328 - accuracy: 0.3415
8/8 [==============================] - 1s 5ms/step - loss: 1.5894 - accuracy: 0.3000
1/1 [==============================] - 0s 186ms/step - loss: 1.5343 - accuracy: 0.4390
8/8 [==============================] - 1s 6ms/step - loss: 1.5922 - accuracy: 0.2216
1/1 [==============================] - 0s 222ms/step - loss: 1.5426 - accuracy: 0.3171
8/8 [==============================] - 0s 4ms/step - loss: 1.5911 - accuracy: 0.2514
1/1 [==============================] - 0s 130ms/step - loss: 1.5773 - accuracy: 0.2927
8/8 [==============================] - 0s 4ms/step - loss: 1.5861 - accuracy: 0.3216
1/1 [==============================] - 0s 132ms/step - loss: 1.6088 - accuracy: 0.2927
8/8 [==============================] - 0s 4ms/step - loss: 1.5929 - accuracy: 0.3054
1/1 [==============================] - 0s 141ms/step - loss: 1.5643 - accuracy: 0.3659
5/5 [==============================] - 1s 4ms/step - loss: 1.6063 - accuracy: 0.2506
best parameters for ANN:  {'batch_size': 100, 'nb_epoch': 20, 'unit': 100}
best score for ANN: 0.3527874529361725

Best Parameters for W2V dataset-

best parameters for ANN: {'batch_size': 100, 'nb_epoch': 20, 'unit': 100}

best score for ANN: 0.3527874529361725

def NN_Model_Tuned_wv(X_train, X_test, y_train, y_test):
  in_dim = X_train.shape[1]

  tf.random.set_seed(7)

  model = Sequential()
  model.add(Dense(50, activation='relu', input_dim = in_dim))
  model.add(Dropout(0.2))
  model.add(Dense(50, activation='relu'))
  model.add(Dense(5 , activation='softmax'))

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=0, patience=3)

  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

  print(model.summary())

  history=model.fit(X_train, y_train_cat, validation_split=0.2, epochs = 20, batch_size = 100, callbacks=[early_stopping])

  train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  result_kfold_df= pd.DataFrame({'model': ['Neural Network'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })



  hist= pd.DataFrame(history.history)

  for col in hist.columns:
    print(col)
    plt.plot(hist[col])
    plt.plot(hist[col])
    plt.title('model-'+col)
    plt.ylabel(col)
    plt.xlabel('epoch')
    plt.show()

  return result_kfold_df

NN_Model_Tuned_wv(X_train_wv, X_test_wv, y_train_wv, y_test_wv)

Model: "sequential_974"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_2928 (Dense)          (None, 50)                10050     
                                                                 
 dropout_12 (Dropout)        (None, 50)                0         
                                                                 
 dense_2929 (Dense)          (None, 50)                2550      
                                                                 
 dense_2930 (Dense)          (None, 5)                 255       
                                                                 
=================================================================
Total params: 12,855
Trainable params: 12,855
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 1s 92ms/step - loss: 1.6075 - accuracy: 0.3321 - val_loss: 1.6029 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 0s 24ms/step - loss: 1.5998 - accuracy: 0.3359 - val_loss: 1.5975 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 0s 14ms/step - loss: 1.5921 - accuracy: 0.3359 - val_loss: 1.5918 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 0s 14ms/step - loss: 1.5856 - accuracy: 0.3359 - val_loss: 1.5860 - val_accuracy: 0.3333
Epoch 5/20
3/3 [==============================] - 0s 14ms/step - loss: 1.5767 - accuracy: 0.3359 - val_loss: 1.5796 - val_accuracy: 0.3333
Epoch 6/20
3/3 [==============================] - 0s 17ms/step - loss: 1.5662 - accuracy: 0.3359 - val_loss: 1.5732 - val_accuracy: 0.3333
Epoch 7/20
3/3 [==============================] - 0s 14ms/step - loss: 1.5565 - accuracy: 0.3359 - val_loss: 1.5666 - val_accuracy: 0.3333
Epoch 8/20
3/3 [==============================] - 0s 13ms/step - loss: 1.5463 - accuracy: 0.3359 - val_loss: 1.5598 - val_accuracy: 0.3333
Epoch 9/20
3/3 [==============================] - 0s 14ms/step - loss: 1.5386 - accuracy: 0.3359 - val_loss: 1.5534 - val_accuracy: 0.3333
Epoch 10/20
3/3 [==============================] - 0s 14ms/step - loss: 1.5244 - accuracy: 0.3359 - val_loss: 1.5474 - val_accuracy: 0.3333
Epoch 11/20
3/3 [==============================] - 0s 14ms/step - loss: 1.5157 - accuracy: 0.3359 - val_loss: 1.5421 - val_accuracy: 0.3333
Epoch 12/20
3/3 [==============================] - 0s 14ms/step - loss: 1.5055 - accuracy: 0.3359 - val_loss: 1.5374 - val_accuracy: 0.3333
Epoch 13/20
3/3 [==============================] - 0s 14ms/step - loss: 1.4939 - accuracy: 0.3359 - val_loss: 1.5338 - val_accuracy: 0.3333
Epoch 14/20
3/3 [==============================] - 0s 14ms/step - loss: 1.4882 - accuracy: 0.3359 - val_loss: 1.5313 - val_accuracy: 0.3333
Epoch 15/20
3/3 [==============================] - 0s 16ms/step - loss: 1.4826 - accuracy: 0.3359 - val_loss: 1.5301 - val_accuracy: 0.3333
Epoch 16/20
3/3 [==============================] - 0s 14ms/step - loss: 1.4744 - accuracy: 0.3359 - val_loss: 1.5302 - val_accuracy: 0.3333
Epoch 17/20
3/3 [==============================] - 0s 14ms/step - loss: 1.4696 - accuracy: 0.3359 - val_loss: 1.5309 - val_accuracy: 0.3333
Epoch 18/20
3/3 [==============================] - 0s 17ms/step - loss: 1.4675 - accuracy: 0.3359 - val_loss: 1.5322 - val_accuracy: 0.3333
11/11 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
11/11 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.335366	0.337349	0.168449	0.170194

NN_Model_Tuned_wv(X_train_wv_smote, X_test_wv, y_train_wv_smote, y_test_wv)

Model: "sequential_975"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_2931 (Dense)          (None, 50)                10050     
                                                                 
 dropout_13 (Dropout)        (None, 50)                0         
                                                                 
 dense_2932 (Dense)          (None, 50)                2550      
                                                                 
 dense_2933 (Dense)          (None, 5)                 255       
                                                                 
=================================================================
Total params: 12,855
Trainable params: 12,855
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 1s 48ms/step - loss: 1.6073 - accuracy: 0.1909 - val_loss: 1.6420 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 0s 9ms/step - loss: 1.5993 - accuracy: 0.2273 - val_loss: 1.6778 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 0s 9ms/step - loss: 1.5904 - accuracy: 0.2295 - val_loss: 1.7192 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 0s 9ms/step - loss: 1.5810 - accuracy: 0.2591 - val_loss: 1.7683 - val_accuracy: 0.0000e+00
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 1ms/step
3/3 [==============================] - 0s 3ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.2	0.337349	0.066667	0.170194

Tuned_ANN(X_wv_fullset, y_wv_fullset)

4/4 [==============================] - 1s 5ms/step - loss: 1.6039 - accuracy: 0.1653
1/1 [==============================] - 0s 323ms/step - loss: 1.5925 - accuracy: 0.1667
4/4 [==============================] - 1s 5ms/step - loss: 1.5982 - accuracy: 0.2757
1/1 [==============================] - 0s 428ms/step - loss: 1.6206 - accuracy: 0.2683
4/4 [==============================] - 2s 5ms/step - loss: 1.5752 - accuracy: 0.3351
1/1 [==============================] - 0s 283ms/step - loss: 1.5024 - accuracy: 0.3902
4/4 [==============================] - 1s 6ms/step - loss: 1.5783 - accuracy: 0.2676
1/1 [==============================] - 0s 168ms/step - loss: 1.6068 - accuracy: 0.1463
4/4 [==============================] - 0s 3ms/step - loss: 1.5644 - accuracy: 0.3324
1/1 [==============================] - 0s 160ms/step - loss: 1.5451 - accuracy: 0.2927
4/4 [==============================] - 0s 3ms/step - loss: 1.5986 - accuracy: 0.2216
1/1 [==============================] - 0s 137ms/step - loss: 1.5702 - accuracy: 0.2683
4/4 [==============================] - 0s 4ms/step - loss: 1.5690 - accuracy: 0.3405
1/1 [==============================] - 0s 132ms/step - loss: 1.5539 - accuracy: 0.3171
4/4 [==============================] - 0s 3ms/step - loss: 1.6214 - accuracy: 0.1514
1/1 [==============================] - 0s 127ms/step - loss: 1.6158 - accuracy: 0.0976
4/4 [==============================] - 0s 4ms/step - loss: 1.6099 - accuracy: 0.1838
1/1 [==============================] - 0s 144ms/step - loss: 1.5832 - accuracy: 0.2683
4/4 [==============================] - 0s 5ms/step - loss: 1.5960 - accuracy: 0.2514
1/1 [==============================] - 0s 125ms/step - loss: 1.5972 - accuracy: 0.3171
4/4 [==============================] - 0s 4ms/step - loss: 1.5795 - accuracy: 0.2981
1/1 [==============================] - 0s 143ms/step - loss: 1.5306 - accuracy: 0.3810
4/4 [==============================] - 0s 4ms/step - loss: 1.6140 - accuracy: 0.2514
1/1 [==============================] - 0s 139ms/step - loss: 1.5944 - accuracy: 0.3171
4/4 [==============================] - 1s 4ms/step - loss: 1.5900 - accuracy: 0.2351
1/1 [==============================] - 0s 131ms/step - loss: 1.5303 - accuracy: 0.3171
4/4 [==============================] - 1s 8ms/step - loss: 1.6024 - accuracy: 0.2378
1/1 [==============================] - 0s 288ms/step - loss: 1.5822 - accuracy: 0.2927
4/4 [==============================] - 1s 10ms/step - loss: 1.5904 - accuracy: 0.2432
1/1 [==============================] - 1s 534ms/step - loss: 1.5840 - accuracy: 0.2439
4/4 [==============================] - 2s 6ms/step - loss: 1.5780 - accuracy: 0.2703
1/1 [==============================] - 0s 408ms/step - loss: 1.5130 - accuracy: 0.3171
4/4 [==============================] - 1s 6ms/step - loss: 1.5852 - accuracy: 0.2676
1/1 [==============================] - 0s 326ms/step - loss: 1.5354 - accuracy: 0.3659
4/4 [==============================] - 1s 7ms/step - loss: 1.5826 - accuracy: 0.3108
1/1 [==============================] - 0s 226ms/step - loss: 1.5793 - accuracy: 0.3171
4/4 [==============================] - 1s 6ms/step - loss: 1.6073 - accuracy: 0.2081
1/1 [==============================] - 0s 201ms/step - loss: 1.6101 - accuracy: 0.0732
4/4 [==============================] - 1s 7ms/step - loss: 1.5930 - accuracy: 0.2054
1/1 [==============================] - 1s 532ms/step - loss: 1.5504 - accuracy: 0.4390
4/4 [==============================] - 0s 4ms/step - loss: 1.5694 - accuracy: 0.2737
1/1 [==============================] - 0s 156ms/step - loss: 1.4973 - accuracy: 0.4286
4/4 [==============================] - 0s 4ms/step - loss: 1.5934 - accuracy: 0.2865
1/1 [==============================] - 0s 144ms/step - loss: 1.5379 - accuracy: 0.2927
4/4 [==============================] - 0s 4ms/step - loss: 1.5958 - accuracy: 0.2595
1/1 [==============================] - 0s 197ms/step - loss: 1.5051 - accuracy: 0.3902
4/4 [==============================] - 1s 7ms/step - loss: 1.5745 - accuracy: 0.2865
1/1 [==============================] - 0s 189ms/step - loss: 1.5412 - accuracy: 0.2439
4/4 [==============================] - 1s 8ms/step - loss: 1.5563 - accuracy: 0.3811
1/1 [==============================] - 0s 215ms/step - loss: 1.4681 - accuracy: 0.3659
4/4 [==============================] - 1s 4ms/step - loss: 1.5481 - accuracy: 0.2919
1/1 [==============================] - 0s 127ms/step - loss: 1.4304 - accuracy: 0.4390
4/4 [==============================] - 0s 4ms/step - loss: 1.5825 - accuracy: 0.2973
1/1 [==============================] - 0s 144ms/step - loss: 1.5211 - accuracy: 0.2683
4/4 [==============================] - 0s 4ms/step - loss: 1.5646 - accuracy: 0.3595
1/1 [==============================] - 0s 135ms/step - loss: 1.5452 - accuracy: 0.3171
4/4 [==============================] - 0s 4ms/step - loss: 1.5742 - accuracy: 0.2568
1/1 [==============================] - 0s 129ms/step - loss: 1.5818 - accuracy: 0.3171
4/4 [==============================] - 0s 4ms/step - loss: 1.5815 - accuracy: 0.3351
1/1 [==============================] - 0s 144ms/step - loss: 1.5236 - accuracy: 0.4146
4/4 [==============================] - 0s 5ms/step - loss: 1.5374 - accuracy: 0.2818
1/1 [==============================] - 0s 143ms/step - loss: 1.4415 - accuracy: 0.4286
4/4 [==============================] - 0s 5ms/step - loss: 1.5316 - accuracy: 0.3054
1/1 [==============================] - 0s 140ms/step - loss: 1.4702 - accuracy: 0.3171
4/4 [==============================] - 0s 9ms/step - loss: 1.5506 - accuracy: 0.3081
1/1 [==============================] - 0s 268ms/step - loss: 1.4118 - accuracy: 0.3902
4/4 [==============================] - 1s 9ms/step - loss: 1.5441 - accuracy: 0.3324
1/1 [==============================] - 0s 245ms/step - loss: 1.5494 - accuracy: 0.2439
4/4 [==============================] - 1s 10ms/step - loss: 1.5566 - accuracy: 0.3459
1/1 [==============================] - 0s 238ms/step - loss: 1.4321 - accuracy: 0.3659
4/4 [==============================] - 1s 6ms/step - loss: 1.5530 - accuracy: 0.3243
1/1 [==============================] - 0s 490ms/step - loss: 1.4256 - accuracy: 0.4146
4/4 [==============================] - 1s 9ms/step - loss: 1.5675 - accuracy: 0.3243
1/1 [==============================] - 0s 451ms/step - loss: 1.4660 - accuracy: 0.3415
4/4 [==============================] - 1s 7ms/step - loss: 1.5365 - accuracy: 0.3459
1/1 [==============================] - 0s 268ms/step - loss: 1.5564 - accuracy: 0.2439
4/4 [==============================] - 1s 9ms/step - loss: 1.5525 - accuracy: 0.3162
1/1 [==============================] - 0s 282ms/step - loss: 1.5498 - accuracy: 0.2683
4/4 [==============================] - 1s 12ms/step - loss: 1.5654 - accuracy: 0.2514
1/1 [==============================] - 0s 350ms/step - loss: 1.4775 - accuracy: 0.4146
4/4 [==============================] - 1s 5ms/step - loss: 1.6135 - accuracy: 0.2602
1/1 [==============================] - 0s 388ms/step - loss: 1.5909 - accuracy: 0.3571
4/4 [==============================] - 2s 6ms/step - loss: 1.5955 - accuracy: 0.3351
1/1 [==============================] - 0s 454ms/step - loss: 1.5719 - accuracy: 0.3171
4/4 [==============================] - 1s 7ms/step - loss: 1.5743 - accuracy: 0.3000
1/1 [==============================] - 1s 529ms/step - loss: 1.5360 - accuracy: 0.3902
4/4 [==============================] - 1s 13ms/step - loss: 1.5717 - accuracy: 0.2784
1/1 [==============================] - 0s 412ms/step - loss: 1.5793 - accuracy: 0.3902
4/4 [==============================] - 1s 3ms/step - loss: 1.5670 - accuracy: 0.3135
1/1 [==============================] - 0s 147ms/step - loss: 1.5424 - accuracy: 0.2927
4/4 [==============================] - 1s 6ms/step - loss: 1.6025 - accuracy: 0.3135
1/1 [==============================] - 0s 230ms/step - loss: 1.5626 - accuracy: 0.3902
4/4 [==============================] - 1s 7ms/step - loss: 1.5892 - accuracy: 0.2622
1/1 [==============================] - 0s 150ms/step - loss: 1.5757 - accuracy: 0.2439
4/4 [==============================] - 1s 7ms/step - loss: 1.6180 - accuracy: 0.2054
1/1 [==============================] - 0s 238ms/step - loss: 1.5763 - accuracy: 0.2927
4/4 [==============================] - 1s 6ms/step - loss: 1.5917 - accuracy: 0.2405
1/1 [==============================] - 0s 478ms/step - loss: 1.6321 - accuracy: 0.1463
4/4 [==============================] - 1s 5ms/step - loss: 1.6038 - accuracy: 0.2216
1/1 [==============================] - 0s 141ms/step - loss: 1.6023 - accuracy: 0.1220
4/4 [==============================] - 1s 4ms/step - loss: 1.6168 - accuracy: 0.2005
1/1 [==============================] - 0s 224ms/step - loss: 1.5932 - accuracy: 0.1429
4/4 [==============================] - 1s 6ms/step - loss: 1.6095 - accuracy: 0.1405
1/1 [==============================] - 0s 271ms/step - loss: 1.5993 - accuracy: 0.1463
4/4 [==============================] - 1s 9ms/step - loss: 1.6065 - accuracy: 0.1811
1/1 [==============================] - 0s 307ms/step - loss: 1.5470 - accuracy: 0.4390
4/4 [==============================] - 1s 6ms/step - loss: 1.6194 - accuracy: 0.1919
1/1 [==============================] - 0s 426ms/step - loss: 1.5545 - accuracy: 0.2439
4/4 [==============================] - 1s 7ms/step - loss: 1.5922 - accuracy: 0.2622
1/1 [==============================] - 0s 206ms/step - loss: 1.5405 - accuracy: 0.2439
4/4 [==============================] - 1s 8ms/step - loss: 1.5967 - accuracy: 0.2405
1/1 [==============================] - 0s 249ms/step - loss: 1.5637 - accuracy: 0.1707
4/4 [==============================] - 1s 7ms/step - loss: 1.5766 - accuracy: 0.2432
1/1 [==============================] - 0s 218ms/step - loss: 1.5184 - accuracy: 0.4146
4/4 [==============================] - 1s 5ms/step - loss: 1.5925 - accuracy: 0.2270
1/1 [==============================] - 0s 215ms/step - loss: 1.5743 - accuracy: 0.2683
4/4 [==============================] - 1s 6ms/step - loss: 1.5977 - accuracy: 0.2919
1/1 [==============================] - 0s 139ms/step - loss: 1.5728 - accuracy: 0.2683
4/4 [==============================] - 0s 4ms/step - loss: 1.6130 - accuracy: 0.2838
1/1 [==============================] - 0s 134ms/step - loss: 1.5846 - accuracy: 0.2683
4/4 [==============================] - 1s 5ms/step - loss: 1.5694 - accuracy: 0.3198
1/1 [==============================] - 0s 156ms/step - loss: 1.5222 - accuracy: 0.3571
4/4 [==============================] - 0s 5ms/step - loss: 1.5855 - accuracy: 0.2541
1/1 [==============================] - 0s 156ms/step - loss: 1.5464 - accuracy: 0.2439
4/4 [==============================] - 0s 5ms/step - loss: 1.5674 - accuracy: 0.3135
1/1 [==============================] - 0s 129ms/step - loss: 1.4609 - accuracy: 0.3659
4/4 [==============================] - 0s 4ms/step - loss: 1.5504 - accuracy: 0.3000
1/1 [==============================] - 0s 138ms/step - loss: 1.5669 - accuracy: 0.2195
4/4 [==============================] - 1s 7ms/step - loss: 1.5867 - accuracy: 0.2838
1/1 [==============================] - 0s 177ms/step - loss: 1.4874 - accuracy: 0.4146
4/4 [==============================] - 1s 5ms/step - loss: 1.5629 - accuracy: 0.3676
1/1 [==============================] - 0s 188ms/step - loss: 1.4806 - accuracy: 0.4390
4/4 [==============================] - 1s 8ms/step - loss: 1.5701 - accuracy: 0.2757
1/1 [==============================] - 0s 213ms/step - loss: 1.4941 - accuracy: 0.4146
4/4 [==============================] - 0s 5ms/step - loss: 1.5684 - accuracy: 0.3000
1/1 [==============================] - 0s 139ms/step - loss: 1.5382 - accuracy: 0.2927
4/4 [==============================] - 1s 7ms/step - loss: 1.5519 - accuracy: 0.3378
1/1 [==============================] - 0s 236ms/step - loss: 1.5839 - accuracy: 0.1951
4/4 [==============================] - 1s 7ms/step - loss: 1.5671 - accuracy: 0.2892
1/1 [==============================] - 0s 221ms/step - loss: 1.4931 - accuracy: 0.3902
4/4 [==============================] - 1s 10ms/step - loss: 1.5763 - accuracy: 0.2547
1/1 [==============================] - 0s 252ms/step - loss: 1.4688 - accuracy: 0.3333
4/4 [==============================] - 1s 10ms/step - loss: 1.5536 - accuracy: 0.3216
1/1 [==============================] - 0s 233ms/step - loss: 1.5036 - accuracy: 0.3171
4/4 [==============================] - 1s 10ms/step - loss: 1.5407 - accuracy: 0.3270
1/1 [==============================] - 0s 269ms/step - loss: 1.4191 - accuracy: 0.3659
4/4 [==============================] - 1s 12ms/step - loss: 1.5691 - accuracy: 0.2784
1/1 [==============================] - 0s 222ms/step - loss: 1.5522 - accuracy: 0.2927
4/4 [==============================] - 1s 14ms/step - loss: 1.5432 - accuracy: 0.3108
1/1 [==============================] - 0s 363ms/step - loss: 1.4195 - accuracy: 0.3415
4/4 [==============================] - 1s 9ms/step - loss: 1.5341 - accuracy: 0.3270
1/1 [==============================] - 0s 406ms/step - loss: 1.3901 - accuracy: 0.4390
4/4 [==============================] - 1s 14ms/step - loss: 1.5682 - accuracy: 0.3297
1/1 [==============================] - 1s 1s/step - loss: 1.4528 - accuracy: 0.3171
4/4 [==============================] - 1s 8ms/step - loss: 1.5493 - accuracy: 0.2811
1/1 [==============================] - 0s 234ms/step - loss: 1.5312 - accuracy: 0.2927
4/4 [==============================] - 1s 10ms/step - loss: 1.5488 - accuracy: 0.2730
1/1 [==============================] - 0s 226ms/step - loss: 1.5429 - accuracy: 0.2439
4/4 [==============================] - 1s 9ms/step - loss: 1.5643 - accuracy: 0.2730
1/1 [==============================] - 0s 235ms/step - loss: 1.4941 - accuracy: 0.3659
19/19 [==============================] - 1s 4ms/step - loss: 1.5754 - accuracy: 0.3388
3/3 [==============================] - 0s 3ms/step - loss: 1.5265 - accuracy: 0.4524
19/19 [==============================] - 1s 4ms/step - loss: 1.5941 - accuracy: 0.2865
3/3 [==============================] - 0s 4ms/step - loss: 1.5641 - accuracy: 0.2439
19/19 [==============================] - 1s 3ms/step - loss: 1.5491 - accuracy: 0.3514
3/3 [==============================] - 0s 6ms/step - loss: 1.4602 - accuracy: 0.3902
19/19 [==============================] - 1s 3ms/step - loss: 1.5413 - accuracy: 0.3486
3/3 [==============================] - 0s 3ms/step - loss: 1.5842 - accuracy: 0.2195
19/19 [==============================] - 0s 2ms/step - loss: 1.5271 - accuracy: 0.3595
3/3 [==============================] - 0s 3ms/step - loss: 1.4668 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5506 - accuracy: 0.2784
3/3 [==============================] - 0s 4ms/step - loss: 1.4475 - accuracy: 0.4390
19/19 [==============================] - 1s 3ms/step - loss: 1.5425 - accuracy: 0.3189
3/3 [==============================] - 0s 4ms/step - loss: 1.4578 - accuracy: 0.2927
19/19 [==============================] - 1s 3ms/step - loss: 1.5899 - accuracy: 0.2216
3/3 [==============================] - 0s 4ms/step - loss: 1.5457 - accuracy: 0.3415
19/19 [==============================] - 1s 3ms/step - loss: 1.5482 - accuracy: 0.3324
3/3 [==============================] - 0s 4ms/step - loss: 1.5590 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5687 - accuracy: 0.3297
3/3 [==============================] - 0s 3ms/step - loss: 1.5470 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.5177 - accuracy: 0.3360
3/3 [==============================] - 0s 3ms/step - loss: 1.4333 - accuracy: 0.4762
19/19 [==============================] - 0s 2ms/step - loss: 1.5555 - accuracy: 0.3000
3/3 [==============================] - 0s 3ms/step - loss: 1.5107 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.5227 - accuracy: 0.3270
3/3 [==============================] - 0s 4ms/step - loss: 1.3740 - accuracy: 0.3902
19/19 [==============================] - 0s 2ms/step - loss: 1.5112 - accuracy: 0.3405
3/3 [==============================] - 0s 3ms/step - loss: 1.5825 - accuracy: 0.2195
19/19 [==============================] - 1s 2ms/step - loss: 1.5522 - accuracy: 0.3459
3/3 [==============================] - 0s 3ms/step - loss: 1.4169 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.5244 - accuracy: 0.3676
3/3 [==============================] - 0s 4ms/step - loss: 1.3649 - accuracy: 0.5610
19/19 [==============================] - 0s 2ms/step - loss: 1.5258 - accuracy: 0.3027
3/3 [==============================] - 0s 3ms/step - loss: 1.4155 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.5022 - accuracy: 0.3378
3/3 [==============================] - 0s 3ms/step - loss: 1.5180 - accuracy: 0.2683
19/19 [==============================] - 0s 2ms/step - loss: 1.5174 - accuracy: 0.3135
3/3 [==============================] - 0s 3ms/step - loss: 1.5625 - accuracy: 0.2683
19/19 [==============================] - 0s 2ms/step - loss: 1.5471 - accuracy: 0.3000
3/3 [==============================] - 0s 4ms/step - loss: 1.4738 - accuracy: 0.3902
19/19 [==============================] - 0s 2ms/step - loss: 1.5015 - accuracy: 0.3089
3/3 [==============================] - 0s 3ms/step - loss: 1.4182 - accuracy: 0.3810
19/19 [==============================] - 1s 3ms/step - loss: 1.4841 - accuracy: 0.3568
3/3 [==============================] - 0s 3ms/step - loss: 1.4513 - accuracy: 0.2927
19/19 [==============================] - 1s 4ms/step - loss: 1.4790 - accuracy: 0.3784
3/3 [==============================] - 0s 4ms/step - loss: 1.3014 - accuracy: 0.3902
19/19 [==============================] - 1s 2ms/step - loss: 1.4709 - accuracy: 0.3514
3/3 [==============================] - 0s 4ms/step - loss: 1.5540 - accuracy: 0.2439
19/19 [==============================] - 0s 3ms/step - loss: 1.5199 - accuracy: 0.3676
3/3 [==============================] - 0s 4ms/step - loss: 1.3605 - accuracy: 0.3171
19/19 [==============================] - 0s 2ms/step - loss: 1.4873 - accuracy: 0.3595
3/3 [==============================] - 0s 4ms/step - loss: 1.3130 - accuracy: 0.5122
19/19 [==============================] - 0s 2ms/step - loss: 1.4956 - accuracy: 0.3486
3/3 [==============================] - 0s 4ms/step - loss: 1.3798 - accuracy: 0.2683
19/19 [==============================] - 0s 3ms/step - loss: 1.4738 - accuracy: 0.3730
3/3 [==============================] - 0s 4ms/step - loss: 1.5011 - accuracy: 0.2195
19/19 [==============================] - 0s 2ms/step - loss: 1.4803 - accuracy: 0.3243
3/3 [==============================] - 0s 3ms/step - loss: 1.5608 - accuracy: 0.2683
19/19 [==============================] - 0s 2ms/step - loss: 1.5027 - accuracy: 0.3162
3/3 [==============================] - 0s 4ms/step - loss: 1.4536 - accuracy: 0.3902
19/19 [==============================] - 0s 3ms/step - loss: 1.4741 - accuracy: 0.2954
3/3 [==============================] - 0s 4ms/step - loss: 1.3996 - accuracy: 0.4524
19/19 [==============================] - 0s 4ms/step - loss: 1.4244 - accuracy: 0.3919
3/3 [==============================] - 0s 4ms/step - loss: 1.4493 - accuracy: 0.2683
19/19 [==============================] - 0s 3ms/step - loss: 1.4432 - accuracy: 0.3676
3/3 [==============================] - 0s 4ms/step - loss: 1.2300 - accuracy: 0.4390
19/19 [==============================] - 0s 3ms/step - loss: 1.4379 - accuracy: 0.3459
3/3 [==============================] - 0s 3ms/step - loss: 1.5142 - accuracy: 0.4390
19/19 [==============================] - 0s 3ms/step - loss: 1.4643 - accuracy: 0.3595
3/3 [==============================] - 0s 4ms/step - loss: 1.2799 - accuracy: 0.3415
19/19 [==============================] - 0s 3ms/step - loss: 1.4738 - accuracy: 0.3162
3/3 [==============================] - 0s 4ms/step - loss: 1.2754 - accuracy: 0.4878
19/19 [==============================] - 1s 4ms/step - loss: 1.4559 - accuracy: 0.3541
3/3 [==============================] - 0s 5ms/step - loss: 1.3784 - accuracy: 0.2927
19/19 [==============================] - 1s 5ms/step - loss: 1.4487 - accuracy: 0.3865
3/3 [==============================] - 0s 5ms/step - loss: 1.5056 - accuracy: 0.3659
19/19 [==============================] - 1s 5ms/step - loss: 1.4685 - accuracy: 0.3459
3/3 [==============================] - 0s 4ms/step - loss: 1.4994 - accuracy: 0.2683
19/19 [==============================] - 0s 4ms/step - loss: 1.4761 - accuracy: 0.3270
3/3 [==============================] - 0s 4ms/step - loss: 1.4133 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5557 - accuracy: 0.2954
3/3 [==============================] - 0s 3ms/step - loss: 1.4979 - accuracy: 0.4762
19/19 [==============================] - 0s 2ms/step - loss: 1.5583 - accuracy: 0.3378
3/3 [==============================] - 0s 3ms/step - loss: 1.5105 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5811 - accuracy: 0.2595
3/3 [==============================] - 0s 3ms/step - loss: 1.5100 - accuracy: 0.4390
19/19 [==============================] - 0s 2ms/step - loss: 1.5539 - accuracy: 0.3216
3/3 [==============================] - 0s 3ms/step - loss: 1.5893 - accuracy: 0.2439
19/19 [==============================] - 0s 2ms/step - loss: 1.5735 - accuracy: 0.3432
3/3 [==============================] - 0s 3ms/step - loss: 1.5260 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.5759 - accuracy: 0.2649
3/3 [==============================] - 0s 3ms/step - loss: 1.5177 - accuracy: 0.2195
19/19 [==============================] - 0s 2ms/step - loss: 1.5572 - accuracy: 0.3027
3/3 [==============================] - 0s 3ms/step - loss: 1.5150 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5992 - accuracy: 0.2919
3/3 [==============================] - 1s 3ms/step - loss: 1.5800 - accuracy: 0.2195
19/19 [==============================] - 0s 2ms/step - loss: 1.5874 - accuracy: 0.3000
3/3 [==============================] - 0s 3ms/step - loss: 1.5803 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5743 - accuracy: 0.3324
3/3 [==============================] - 0s 4ms/step - loss: 1.5567 - accuracy: 0.3659
19/19 [==============================] - 0s 2ms/step - loss: 1.5542 - accuracy: 0.2575
3/3 [==============================] - 0s 5ms/step - loss: 1.4833 - accuracy: 0.3810
19/19 [==============================] - 1s 4ms/step - loss: 1.5268 - accuracy: 0.3405
3/3 [==============================] - 0s 4ms/step - loss: 1.5135 - accuracy: 0.3659
19/19 [==============================] - 1s 3ms/step - loss: 1.5367 - accuracy: 0.3351
3/3 [==============================] - 0s 6ms/step - loss: 1.4068 - accuracy: 0.4146
19/19 [==============================] - 1s 2ms/step - loss: 1.5083 - accuracy: 0.3378
3/3 [==============================] - 0s 4ms/step - loss: 1.5817 - accuracy: 0.2195
19/19 [==============================] - 0s 2ms/step - loss: 1.5139 - accuracy: 0.3297
3/3 [==============================] - 0s 3ms/step - loss: 1.3783 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.5510 - accuracy: 0.2757
3/3 [==============================] - 0s 3ms/step - loss: 1.4125 - accuracy: 0.4390
19/19 [==============================] - 0s 2ms/step - loss: 1.5241 - accuracy: 0.3622
3/3 [==============================] - 0s 3ms/step - loss: 1.4252 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5312 - accuracy: 0.3081
3/3 [==============================] - 0s 4ms/step - loss: 1.5102 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.5148 - accuracy: 0.3297
3/3 [==============================] - 0s 6ms/step - loss: 1.5797 - accuracy: 0.2439
19/19 [==============================] - 1s 2ms/step - loss: 1.5340 - accuracy: 0.3297
3/3 [==============================] - 0s 3ms/step - loss: 1.4588 - accuracy: 0.3902
19/19 [==============================] - 0s 2ms/step - loss: 1.5049 - accuracy: 0.2900
3/3 [==============================] - 0s 3ms/step - loss: 1.4103 - accuracy: 0.3571
19/19 [==============================] - 0s 3ms/step - loss: 1.4825 - accuracy: 0.3297
3/3 [==============================] - 0s 3ms/step - loss: 1.4713 - accuracy: 0.3415
19/19 [==============================] - 0s 2ms/step - loss: 1.4999 - accuracy: 0.3216
3/3 [==============================] - 0s 4ms/step - loss: 1.3055 - accuracy: 0.3902
19/19 [==============================] - 0s 2ms/step - loss: 1.4731 - accuracy: 0.3297
3/3 [==============================] - 0s 3ms/step - loss: 1.5352 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.4856 - accuracy: 0.3324
3/3 [==============================] - 0s 4ms/step - loss: 1.3282 - accuracy: 0.4146
19/19 [==============================] - 0s 2ms/step - loss: 1.4827 - accuracy: 0.3622
3/3 [==============================] - 0s 4ms/step - loss: 1.2980 - accuracy: 0.4634
19/19 [==============================] - 1s 4ms/step - loss: 1.4966 - accuracy: 0.3108
3/3 [==============================] - 0s 3ms/step - loss: 1.3835 - accuracy: 0.2683
19/19 [==============================] - 1s 4ms/step - loss: 1.4742 - accuracy: 0.3378
3/3 [==============================] - 0s 4ms/step - loss: 1.5075 - accuracy: 0.2683
19/19 [==============================] - 1s 2ms/step - loss: 1.4889 - accuracy: 0.3081
3/3 [==============================] - 0s 4ms/step - loss: 1.5672 - accuracy: 0.2927
19/19 [==============================] - 0s 2ms/step - loss: 1.4924 - accuracy: 0.2973
3/3 [==============================] - 0s 4ms/step - loss: 1.4097 - accuracy: 0.3902
19/19 [==============================] - 0s 3ms/step - loss: 1.4718 - accuracy: 0.2900
3/3 [==============================] - 0s 5ms/step - loss: 1.3953 - accuracy: 0.4524
19/19 [==============================] - 0s 3ms/step - loss: 1.4437 - accuracy: 0.3757
3/3 [==============================] - 0s 4ms/step - loss: 1.4541 - accuracy: 0.2927
19/19 [==============================] - 0s 3ms/step - loss: 1.4369 - accuracy: 0.3649
3/3 [==============================] - 0s 4ms/step - loss: 1.2418 - accuracy: 0.4390
19/19 [==============================] - 0s 3ms/step - loss: 1.4519 - accuracy: 0.3243
3/3 [==============================] - 0s 4ms/step - loss: 1.5196 - accuracy: 0.2927
19/19 [==============================] - 1s 5ms/step - loss: 1.4570 - accuracy: 0.3405
3/3 [==============================] - 0s 4ms/step - loss: 1.2601 - accuracy: 0.4146
19/19 [==============================] - 0s 3ms/step - loss: 1.4543 - accuracy: 0.3405
3/3 [==============================] - 0s 4ms/step - loss: 1.2670 - accuracy: 0.4634
19/19 [==============================] - 0s 3ms/step - loss: 1.4652 - accuracy: 0.3541
3/3 [==============================] - 0s 4ms/step - loss: 1.3686 - accuracy: 0.2927
19/19 [==============================] - 0s 3ms/step - loss: 1.4569 - accuracy: 0.3865
3/3 [==============================] - 0s 5ms/step - loss: 1.4850 - accuracy: 0.3415
19/19 [==============================] - 0s 3ms/step - loss: 1.4731 - accuracy: 0.3432
3/3 [==============================] - 0s 4ms/step - loss: 1.5551 - accuracy: 0.2683
19/19 [==============================] - 0s 3ms/step - loss: 1.4707 - accuracy: 0.3189
3/3 [==============================] - 0s 4ms/step - loss: 1.4197 - accuracy: 0.3659
8/8 [==============================] - 0s 3ms/step - loss: 1.5814 - accuracy: 0.2276
1/1 [==============================] - 0s 161ms/step - loss: 1.5396 - accuracy: 0.3095
8/8 [==============================] - 2s 3ms/step - loss: 1.5810 - accuracy: 0.2892
1/1 [==============================] - 0s 245ms/step - loss: 1.5816 - accuracy: 0.3415
8/8 [==============================] - 1s 3ms/step - loss: 1.5612 - accuracy: 0.3297
1/1 [==============================] - 0s 157ms/step - loss: 1.5094 - accuracy: 0.3902
8/8 [==============================] - 0s 3ms/step - loss: 1.6109 - accuracy: 0.2189
1/1 [==============================] - 0s 145ms/step - loss: 1.5968 - accuracy: 0.1707
8/8 [==============================] - 0s 3ms/step - loss: 1.6069 - accuracy: 0.2324
1/1 [==============================] - 0s 137ms/step - loss: 1.5996 - accuracy: 0.2439
8/8 [==============================] - 0s 2ms/step - loss: 1.6207 - accuracy: 0.1703
1/1 [==============================] - 0s 138ms/step - loss: 1.5796 - accuracy: 0.4390
8/8 [==============================] - 0s 2ms/step - loss: 1.5826 - accuracy: 0.2189
1/1 [==============================] - 0s 132ms/step - loss: 1.5044 - accuracy: 0.4390
8/8 [==============================] - 0s 3ms/step - loss: 1.5882 - accuracy: 0.3351
1/1 [==============================] - 0s 137ms/step - loss: 1.5897 - accuracy: 0.2683
8/8 [==============================] - 0s 3ms/step - loss: 1.5789 - accuracy: 0.2486
1/1 [==============================] - 0s 146ms/step - loss: 1.6026 - accuracy: 0.2683
8/8 [==============================] - 0s 3ms/step - loss: 1.6175 - accuracy: 0.1865
1/1 [==============================] - 0s 149ms/step - loss: 1.5944 - accuracy: 0.1951
8/8 [==============================] - 0s 3ms/step - loss: 1.5366 - accuracy: 0.3198
1/1 [==============================] - 0s 132ms/step - loss: 1.4755 - accuracy: 0.4286
8/8 [==============================] - 0s 3ms/step - loss: 1.5594 - accuracy: 0.3270
1/1 [==============================] - 0s 133ms/step - loss: 1.5169 - accuracy: 0.3171
8/8 [==============================] - 0s 3ms/step - loss: 1.5532 - accuracy: 0.2892
1/1 [==============================] - 0s 126ms/step - loss: 1.4567 - accuracy: 0.4390
8/8 [==============================] - 0s 3ms/step - loss: 1.5856 - accuracy: 0.2595
1/1 [==============================] - 0s 158ms/step - loss: 1.5459 - accuracy: 0.4390
8/8 [==============================] - 0s 3ms/step - loss: 1.5568 - accuracy: 0.3189
1/1 [==============================] - 0s 139ms/step - loss: 1.4660 - accuracy: 0.3659
8/8 [==============================] - 0s 3ms/step - loss: 1.5386 - accuracy: 0.2973
1/1 [==============================] - 0s 191ms/step - loss: 1.4239 - accuracy: 0.4146
8/8 [==============================] - 1s 3ms/step - loss: 1.5720 - accuracy: 0.2811
1/1 [==============================] - 0s 194ms/step - loss: 1.4957 - accuracy: 0.3659
8/8 [==============================] - 1s 5ms/step - loss: 1.5920 - accuracy: 0.2216
1/1 [==============================] - 0s 190ms/step - loss: 1.5770 - accuracy: 0.1951
8/8 [==============================] - 1s 3ms/step - loss: 1.5593 - accuracy: 0.2973
1/1 [==============================] - 0s 133ms/step - loss: 1.5781 - accuracy: 0.1951
8/8 [==============================] - 0s 3ms/step - loss: 1.5594 - accuracy: 0.3622
1/1 [==============================] - 0s 135ms/step - loss: 1.5155 - accuracy: 0.3659
8/8 [==============================] - 0s 5ms/step - loss: 1.5646 - accuracy: 0.2547
1/1 [==============================] - 0s 161ms/step - loss: 1.4654 - accuracy: 0.3571
8/8 [==============================] - 0s 3ms/step - loss: 1.5333 - accuracy: 0.3027
1/1 [==============================] - 0s 154ms/step - loss: 1.4839 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5302 - accuracy: 0.3162
1/1 [==============================] - 0s 126ms/step - loss: 1.3992 - accuracy: 0.3902
8/8 [==============================] - 0s 3ms/step - loss: 1.5489 - accuracy: 0.3162
1/1 [==============================] - 0s 134ms/step - loss: 1.5120 - accuracy: 0.2927
8/8 [==============================] - 0s 5ms/step - loss: 1.5351 - accuracy: 0.3568
1/1 [==============================] - 0s 140ms/step - loss: 1.3963 - accuracy: 0.3659
8/8 [==============================] - 0s 3ms/step - loss: 1.5447 - accuracy: 0.3000
1/1 [==============================] - 0s 133ms/step - loss: 1.3990 - accuracy: 0.4390
8/8 [==============================] - 0s 3ms/step - loss: 1.5402 - accuracy: 0.2973
1/1 [==============================] - 0s 128ms/step - loss: 1.4430 - accuracy: 0.3171
8/8 [==============================] - 0s 3ms/step - loss: 1.5317 - accuracy: 0.3892
1/1 [==============================] - 0s 120ms/step - loss: 1.5231 - accuracy: 0.2683
8/8 [==============================] - 0s 3ms/step - loss: 1.5202 - accuracy: 0.3243
1/1 [==============================] - 0s 158ms/step - loss: 1.5771 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.5408 - accuracy: 0.3108
1/1 [==============================] - 0s 127ms/step - loss: 1.4530 - accuracy: 0.3659
8/8 [==============================] - 0s 4ms/step - loss: 1.5138 - accuracy: 0.3008
1/1 [==============================] - 0s 143ms/step - loss: 1.4185 - accuracy: 0.3571
8/8 [==============================] - 0s 4ms/step - loss: 1.5114 - accuracy: 0.3622
1/1 [==============================] - 0s 226ms/step - loss: 1.4655 - accuracy: 0.2683
8/8 [==============================] - 1s 6ms/step - loss: 1.5178 - accuracy: 0.3270
1/1 [==============================] - 0s 175ms/step - loss: 1.3475 - accuracy: 0.3902
8/8 [==============================] - 1s 6ms/step - loss: 1.5349 - accuracy: 0.3081
1/1 [==============================] - 0s 207ms/step - loss: 1.5537 - accuracy: 0.2195
8/8 [==============================] - 1s 3ms/step - loss: 1.5002 - accuracy: 0.3189
1/1 [==============================] - 0s 121ms/step - loss: 1.3481 - accuracy: 0.3659
8/8 [==============================] - 1s 4ms/step - loss: 1.5184 - accuracy: 0.3189
1/1 [==============================] - 0s 153ms/step - loss: 1.3366 - accuracy: 0.3902
8/8 [==============================] - 0s 4ms/step - loss: 1.5223 - accuracy: 0.3649
1/1 [==============================] - 0s 134ms/step - loss: 1.4057 - accuracy: 0.2683
8/8 [==============================] - 0s 5ms/step - loss: 1.5225 - accuracy: 0.3378
1/1 [==============================] - 0s 135ms/step - loss: 1.5179 - accuracy: 0.2683
8/8 [==============================] - 0s 4ms/step - loss: 1.5153 - accuracy: 0.3432
1/1 [==============================] - 0s 134ms/step - loss: 1.5797 - accuracy: 0.2683
8/8 [==============================] - 0s 4ms/step - loss: 1.5144 - accuracy: 0.3162
1/1 [==============================] - 0s 159ms/step - loss: 1.4271 - accuracy: 0.3659
8/8 [==============================] - 0s 3ms/step - loss: 1.6063 - accuracy: 0.2493
1/1 [==============================] - 0s 139ms/step - loss: 1.5841 - accuracy: 0.3095
8/8 [==============================] - 0s 3ms/step - loss: 1.5562 - accuracy: 0.2514
1/1 [==============================] - 0s 131ms/step - loss: 1.5649 - accuracy: 0.2195
8/8 [==============================] - 0s 3ms/step - loss: 1.5796 - accuracy: 0.2676
1/1 [==============================] - 0s 141ms/step - loss: 1.5048 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.6031 - accuracy: 0.2216
1/1 [==============================] - 0s 149ms/step - loss: 1.6115 - accuracy: 0.1220
8/8 [==============================] - 0s 3ms/step - loss: 1.5811 - accuracy: 0.3351
1/1 [==============================] - 0s 135ms/step - loss: 1.5545 - accuracy: 0.3659
8/8 [==============================] - 0s 3ms/step - loss: 1.6077 - accuracy: 0.2838
1/1 [==============================] - 0s 125ms/step - loss: 1.5492 - accuracy: 0.4390
8/8 [==============================] - 0s 3ms/step - loss: 1.5546 - accuracy: 0.3243
1/1 [==============================] - 0s 193ms/step - loss: 1.5137 - accuracy: 0.3171
8/8 [==============================] - 1s 4ms/step - loss: 1.5492 - accuracy: 0.3135
1/1 [==============================] - 0s 194ms/step - loss: 1.5609 - accuracy: 0.3415
8/8 [==============================] - 1s 4ms/step - loss: 1.6287 - accuracy: 0.1973
1/1 [==============================] - 0s 204ms/step - loss: 1.6024 - accuracy: 0.1707
8/8 [==============================] - 1s 3ms/step - loss: 1.5606 - accuracy: 0.2784
1/1 [==============================] - 0s 132ms/step - loss: 1.5333 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5793 - accuracy: 0.2304
1/1 [==============================] - 0s 133ms/step - loss: 1.5272 - accuracy: 0.3571
8/8 [==============================] - 0s 3ms/step - loss: 1.5783 - accuracy: 0.3081
1/1 [==============================] - 0s 130ms/step - loss: 1.5296 - accuracy: 0.2439
8/8 [==============================] - 0s 3ms/step - loss: 1.6005 - accuracy: 0.2514
1/1 [==============================] - 0s 148ms/step - loss: 1.5352 - accuracy: 0.4146
8/8 [==============================] - 0s 3ms/step - loss: 1.5763 - accuracy: 0.3378
1/1 [==============================] - 0s 132ms/step - loss: 1.5658 - accuracy: 0.1463
8/8 [==============================] - 0s 3ms/step - loss: 1.5521 - accuracy: 0.3189
1/1 [==============================] - 0s 133ms/step - loss: 1.4292 - accuracy: 0.3659
8/8 [==============================] - 0s 3ms/step - loss: 1.5730 - accuracy: 0.2351
1/1 [==============================] - 0s 140ms/step - loss: 1.4966 - accuracy: 0.1951
8/8 [==============================] - 0s 3ms/step - loss: 1.5699 - accuracy: 0.3054
1/1 [==============================] - 0s 144ms/step - loss: 1.5021 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5519 - accuracy: 0.3784
1/1 [==============================] - 0s 130ms/step - loss: 1.5369 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.5645 - accuracy: 0.3081
1/1 [==============================] - 0s 128ms/step - loss: 1.5756 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.5447 - accuracy: 0.3270
1/1 [==============================] - 0s 133ms/step - loss: 1.5162 - accuracy: 0.3415
8/8 [==============================] - 0s 3ms/step - loss: 1.5451 - accuracy: 0.2791
1/1 [==============================] - 0s 128ms/step - loss: 1.4622 - accuracy: 0.3571
8/8 [==============================] - 0s 4ms/step - loss: 1.5162 - accuracy: 0.3405
1/1 [==============================] - 0s 131ms/step - loss: 1.5088 - accuracy: 0.2683
8/8 [==============================] - 0s 4ms/step - loss: 1.5544 - accuracy: 0.3081
1/1 [==============================] - 0s 182ms/step - loss: 1.4093 - accuracy: 0.3902
8/8 [==============================] - 1s 5ms/step - loss: 1.5275 - accuracy: 0.3378
1/1 [==============================] - 0s 176ms/step - loss: 1.5688 - accuracy: 0.2195
8/8 [==============================] - 1s 4ms/step - loss: 1.5429 - accuracy: 0.3595
1/1 [==============================] - 0s 212ms/step - loss: 1.4324 - accuracy: 0.3902
8/8 [==============================] - 1s 6ms/step - loss: 1.5319 - accuracy: 0.3649
1/1 [==============================] - 0s 128ms/step - loss: 1.3916 - accuracy: 0.4146
8/8 [==============================] - 1s 4ms/step - loss: 1.5426 - accuracy: 0.3216
1/1 [==============================] - 0s 328ms/step - loss: 1.4414 - accuracy: 0.3171
8/8 [==============================] - 0s 4ms/step - loss: 1.5175 - accuracy: 0.3243
1/1 [==============================] - 0s 137ms/step - loss: 1.5086 - accuracy: 0.2927
8/8 [==============================] - 0s 3ms/step - loss: 1.5558 - accuracy: 0.2459
1/1 [==============================] - 1s 956ms/step - loss: 1.5521 - accuracy: 0.2195
8/8 [==============================] - 0s 3ms/step - loss: 1.5188 - accuracy: 0.3270
1/1 [==============================] - 0s 153ms/step - loss: 1.4307 - accuracy: 0.3659
8/8 [==============================] - 0s 4ms/step - loss: 1.5059 - accuracy: 0.3008
1/1 [==============================] - 0s 136ms/step - loss: 1.4160 - accuracy: 0.3571
8/8 [==============================] - 0s 4ms/step - loss: 1.4885 - accuracy: 0.3838
1/1 [==============================] - 0s 133ms/step - loss: 1.4707 - accuracy: 0.2439
8/8 [==============================] - 0s 4ms/step - loss: 1.4998 - accuracy: 0.3703
1/1 [==============================] - 0s 126ms/step - loss: 1.3190 - accuracy: 0.3902
8/8 [==============================] - 0s 4ms/step - loss: 1.4786 - accuracy: 0.3514
1/1 [==============================] - 0s 147ms/step - loss: 1.5792 - accuracy: 0.2439
8/8 [==============================] - 0s 4ms/step - loss: 1.5185 - accuracy: 0.2946
1/1 [==============================] - 0s 129ms/step - loss: 1.3457 - accuracy: 0.3415
8/8 [==============================] - 0s 4ms/step - loss: 1.4961 - accuracy: 0.3189
1/1 [==============================] - 0s 160ms/step - loss: 1.3214 - accuracy: 0.4146
8/8 [==============================] - 1s 7ms/step - loss: 1.5155 - accuracy: 0.3135
1/1 [==============================] - 0s 215ms/step - loss: 1.4034 - accuracy: 0.3171
8/8 [==============================] - 1s 6ms/step - loss: 1.5108 - accuracy: 0.3676
1/1 [==============================] - 0s 192ms/step - loss: 1.5043 - accuracy: 0.2927
8/8 [==============================] - 1s 7ms/step - loss: 1.5168 - accuracy: 0.3162
1/1 [==============================] - 0s 208ms/step - loss: 1.5492 - accuracy: 0.2683
8/8 [==============================] - 0s 4ms/step - loss: 1.5406 - accuracy: 0.2703
1/1 [==============================] - 0s 134ms/step - loss: 1.4203 - accuracy: 0.3902
21/21 [==============================] - 1s 4ms/step - loss: 1.4530 - accuracy: 0.3382
best parameters for ANN:  {'batch_size': 20, 'nb_epoch': 20, 'unit': 300}
best score for ANN: 0.36962834298610686

Best parameters for Word2vec full dataset-

best parameters for ANN: {'batch_size': 20, 'nb_epoch': 20, 'unit': 300}

best score for ANN: 0.36962834298610686

def NN_Model_Tuned_wvfull(X_train, X_test, y_train, y_test):
  in_dim = X_train.shape[1]

  tf.random.set_seed(7)

  model = Sequential()
  model.add(Dense(150, activation='relu', input_dim = in_dim))
  model.add(Dropout(0.2))
  model.add(Dense(150, activation='relu'))
  model.add(Dense(5 , activation='softmax'))

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=0, patience=3)

  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

  print(model.summary())

  history=model.fit(X_train, y_train_cat, validation_split=0.2, epochs = 20, batch_size = 20, callbacks=[early_stopping])

  train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  result_kfold_df= pd.DataFrame({'model': ['Neural Network'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })

  # plotting the model architecture
  from tensorflow.keras.utils import plot_model
  plot_model(model, to_file='/content/drive/MyDrive/Colab Notebooks/Projects/NLP/ann_wv_model.png', show_shapes=True, show_layer_names=True)


  # Confusion matrix for the best model
  cm = confusion_matrix(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))

  fig, ax = plt.subplots(nrows = 1, ncols = 1, figsize = (5,4))
  sns.heatmap(cm,
                annot=True,
                fmt='g',
                xticklabels=['I','II','III','IV','V'],
                yticklabels=['I','II','III','IV','V'], ax = ax)
  ax.set_ylabel('Prediction', fontsize = 11)
  ax.set_xlabel('Actual',fontsize = 11)
  ax.set_title(f'Confusion Matrix: ANN model',fontsize = 14)
  plt.show()

  hist= pd.DataFrame(history.history)

  for col in hist.columns:
    print(col)
    plt.plot(hist[col])
    plt.plot(hist[col])
    plt.title('model-'+col)
    plt.ylabel(col)
    plt.xlabel('epoch')
    plt.show()

  # saving the model
  model.save("/content/drive/MyDrive/Colab Notebooks/Projects/NLP/ANN_wv_model.h5")

  return result_kfold_df

NN_Model_Tuned_wvfull(X_train_wvfull, X_test_wvfull, y_train_wvfull, y_test_wvfull)

Model: "sequential_7"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_7 (Dense)             (None, 150)               33000     
                                                                 
 dropout (Dropout)           (None, 150)               0         
                                                                 
 dense_8 (Dense)             (None, 150)               22650     
                                                                 
 dense_9 (Dense)             (None, 5)                 755       
                                                                 
=================================================================
Total params: 56,405
Trainable params: 56,405
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
14/14 [==============================] - 1s 25ms/step - loss: 1.5509 - accuracy: 0.2748 - val_loss: 1.4939 - val_accuracy: 0.3636
Epoch 2/20
14/14 [==============================] - 0s 9ms/step - loss: 1.4219 - accuracy: 0.3588 - val_loss: 1.4317 - val_accuracy: 0.3636
Epoch 3/20
14/14 [==============================] - 0s 6ms/step - loss: 1.3476 - accuracy: 0.4198 - val_loss: 1.4073 - val_accuracy: 0.3333
Epoch 4/20
14/14 [==============================] - 0s 5ms/step - loss: 1.3083 - accuracy: 0.4427 - val_loss: 1.3922 - val_accuracy: 0.3788
Epoch 5/20
14/14 [==============================] - 0s 7ms/step - loss: 1.2655 - accuracy: 0.4580 - val_loss: 1.3606 - val_accuracy: 0.3939
Epoch 6/20
14/14 [==============================] - 0s 9ms/step - loss: 1.2473 - accuracy: 0.4618 - val_loss: 1.3534 - val_accuracy: 0.4091
Epoch 7/20
14/14 [==============================] - 0s 7ms/step - loss: 1.2313 - accuracy: 0.4809 - val_loss: 1.3269 - val_accuracy: 0.4091
Epoch 8/20
14/14 [==============================] - 0s 7ms/step - loss: 1.2227 - accuracy: 0.4542 - val_loss: 1.3252 - val_accuracy: 0.4091
Epoch 9/20
14/14 [==============================] - 0s 8ms/step - loss: 1.1990 - accuracy: 0.4695 - val_loss: 1.3384 - val_accuracy: 0.3939
Epoch 10/20
14/14 [==============================] - 0s 8ms/step - loss: 1.1841 - accuracy: 0.5000 - val_loss: 1.3247 - val_accuracy: 0.3939
Epoch 11/20
14/14 [==============================] - 0s 8ms/step - loss: 1.1931 - accuracy: 0.4656 - val_loss: 1.3366 - val_accuracy: 0.3788
Epoch 12/20
14/14 [==============================] - 0s 7ms/step - loss: 1.1787 - accuracy: 0.4733 - val_loss: 1.3276 - val_accuracy: 0.3788
Epoch 13/20
14/14 [==============================] - 0s 8ms/step - loss: 1.1754 - accuracy: 0.4847 - val_loss: 1.3469 - val_accuracy: 0.3636
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 5ms/step
11/11 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
11/11 [==============================] - 0s 2ms/step

loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.463415	0.385542	0.442168	0.368834

from PIL import Image
from IPython.display import display

img = Image.open('/content/drive/MyDrive/Colab Notebooks/Projects/NLP/ann_wv_model.png')

display(img)

NN_Model_Tuned_wvfull(X_train_wvfull_smote, X_test_wvfull, y_train_wvfull_smote, y_test_wvfull)

Model: "sequential_1218"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_3660 (Dense)          (None, 150)               33000     
                                                                 
 dropout_15 (Dropout)        (None, 150)               0         
                                                                 
 dense_3661 (Dense)          (None, 150)               22650     
                                                                 
 dense_3662 (Dense)          (None, 5)                 755       
                                                                 
=================================================================
Total params: 56,405
Trainable params: 56,405
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
22/22 [==============================] - 1s 17ms/step - loss: 1.4910 - accuracy: 0.3523 - val_loss: 2.1000 - val_accuracy: 0.0000e+00
Epoch 2/20
22/22 [==============================] - 0s 5ms/step - loss: 1.2893 - accuracy: 0.4773 - val_loss: 2.4495 - val_accuracy: 0.0000e+00
Epoch 3/20
22/22 [==============================] - 0s 5ms/step - loss: 1.2100 - accuracy: 0.4705 - val_loss: 2.2196 - val_accuracy: 0.0636
Epoch 4/20
22/22 [==============================] - 0s 6ms/step - loss: 1.1612 - accuracy: 0.4977 - val_loss: 2.1167 - val_accuracy: 0.0545
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 3ms/step
18/18 [==============================] - 0s 2ms/step
3/3 [==============================] - 0s 4ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.427273	0.433735	0.376974	0.398936

Observations-

i. Smote datasets are better fit. Test accuracy is best around 43%. Best test F1 score is around 40% in Word2Vec technique for full dataset.

ii. We have observed that word2vec is best dataset after applying smote technique.

iii. Out of all datasets, best accuracy in ANN model is 44%. Best F1 score is 40%. This is with word2vec dataset.

iv. Best model is saved to "/content/drive/MyDrive/Colab Notebooks/Projects/NLP/ANN_wv_model.h5".

LSTM Section-

Let's create LSTM models and apply with Countvectorizer, TFIDF and Word2vec datasets.

CountVectorizer-

LSTM_Model(X_train_cv, X_test_cv, y_train_cv, y_test_cv)

Model: "sequential_1220"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding (Embedding)       (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d (SpatialD  (None, 200, 16)          0         
 ropout1D)                                                       
                                                                 
 lstm (LSTM)                 (None, 200)               173600    
                                                                 
 dense_3663 (Dense)          (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 17s 879ms/step - loss: 1.5490 - accuracy: 0.2939 - val_loss: 1.5531 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 12s 798ms/step - loss: 1.5182 - accuracy: 0.3359 - val_loss: 1.5387 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 14s 1s/step - loss: 1.4756 - accuracy: 0.2977 - val_loss: 1.5461 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 12s 844ms/step - loss: 1.4736 - accuracy: 0.3359 - val_loss: 1.5473 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 13s 898ms/step - loss: 1.4622 - accuracy: 0.3359 - val_loss: 1.5740 - val_accuracy: 0.3333
11/11 [==============================] - 1s 93ms/step
3/3 [==============================] - 0s 86ms/step
11/11 [==============================] - 1s 90ms/step
3/3 [==============================] - 0s 88ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.335366	0.337349	0.168449	0.170194

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(5):
  result=LSTM_Model(X_train_cv, X_test_cv, y_train_cv, y_test_cv)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_1221"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1 (Embedding)     (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_1 (Spatia  (None, 200, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_1 (LSTM)               (None, 200)               173600    
                                                                 
 dense_3664 (Dense)          (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 16s 895ms/step - loss: 1.5735 - accuracy: 0.2672 - val_loss: 1.5475 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 12s 815ms/step - loss: 1.5312 - accuracy: 0.3359 - val_loss: 1.5537 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 12s 841ms/step - loss: 1.4961 - accuracy: 0.2595 - val_loss: 1.5770 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 13s 898ms/step - loss: 1.4911 - accuracy: 0.3282 - val_loss: 1.5395 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 13s 907ms/step - loss: 1.4697 - accuracy: 0.3359 - val_loss: 1.5468 - val_accuracy: 0.3333
Epoch 6/100
14/14 [==============================] - 14s 1s/step - loss: 1.4654 - accuracy: 0.2786 - val_loss: 1.5843 - val_accuracy: 0.2727
Epoch 7/100
14/14 [==============================] - 13s 902ms/step - loss: 1.4667 - accuracy: 0.3397 - val_loss: 1.5505 - val_accuracy: 0.3333
11/11 [==============================] - 1s 90ms/step
3/3 [==============================] - 0s 90ms/step
11/11 [==============================] - 1s 90ms/step
3/3 [==============================] - 0s 95ms/step
Model: "sequential_1222"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_2 (Embedding)     (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_2 (Spatia  (None, 200, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_2 (LSTM)               (None, 200)               173600    
                                                                 
 dense_3665 (Dense)          (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 16s 900ms/step - loss: 1.5607 - accuracy: 0.2786 - val_loss: 1.5416 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 11s 797ms/step - loss: 1.4963 - accuracy: 0.3359 - val_loss: 1.5348 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 12s 841ms/step - loss: 1.4700 - accuracy: 0.3359 - val_loss: 1.5627 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 13s 903ms/step - loss: 1.4776 - accuracy: 0.2901 - val_loss: 1.5461 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 13s 913ms/step - loss: 1.4616 - accuracy: 0.3359 - val_loss: 1.5701 - val_accuracy: 0.3333
11/11 [==============================] - 1s 88ms/step
3/3 [==============================] - 0s 88ms/step
11/11 [==============================] - 1s 89ms/step
3/3 [==============================] - 0s 97ms/step
Model: "sequential_1223"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_3 (Embedding)     (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_3 (Spatia  (None, 200, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_3 (LSTM)               (None, 200)               173600    
                                                                 
 dense_3666 (Dense)          (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 15s 876ms/step - loss: 1.5823 - accuracy: 0.2939 - val_loss: 1.5490 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 13s 903ms/step - loss: 1.5368 - accuracy: 0.2786 - val_loss: 1.5598 - val_accuracy: 0.2121
Epoch 3/100
14/14 [==============================] - 13s 905ms/step - loss: 1.5038 - accuracy: 0.2481 - val_loss: 1.5697 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 13s 903ms/step - loss: 1.4974 - accuracy: 0.3092 - val_loss: 1.5362 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 14s 1s/step - loss: 1.4742 - accuracy: 0.3359 - val_loss: 1.5426 - val_accuracy: 0.3333
Epoch 6/100
14/14 [==============================] - 12s 899ms/step - loss: 1.4659 - accuracy: 0.2977 - val_loss: 1.5826 - val_accuracy: 0.3333
Epoch 7/100
14/14 [==============================] - 13s 905ms/step - loss: 1.4662 - accuracy: 0.3359 - val_loss: 1.5531 - val_accuracy: 0.3333
11/11 [==============================] - 1s 101ms/step
3/3 [==============================] - 0s 130ms/step
11/11 [==============================] - 2s 139ms/step
3/3 [==============================] - 0s 95ms/step
Model: "sequential_1224"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_4 (Embedding)     (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_4 (Spatia  (None, 200, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_4 (LSTM)               (None, 200)               173600    
                                                                 
 dense_3667 (Dense)          (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 16s 946ms/step - loss: 1.5692 - accuracy: 0.2672 - val_loss: 1.5444 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 12s 899ms/step - loss: 1.5306 - accuracy: 0.3359 - val_loss: 1.5500 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 12s 900ms/step - loss: 1.4943 - accuracy: 0.3321 - val_loss: 1.5650 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 12s 857ms/step - loss: 1.4930 - accuracy: 0.2672 - val_loss: 1.5438 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 12s 786ms/step - loss: 1.4699 - accuracy: 0.3359 - val_loss: 1.5512 - val_accuracy: 0.3333
Epoch 6/100
14/14 [==============================] - 12s 847ms/step - loss: 1.4636 - accuracy: 0.3015 - val_loss: 1.5877 - val_accuracy: 0.3333
Epoch 7/100
14/14 [==============================] - 14s 1s/step - loss: 1.4690 - accuracy: 0.3397 - val_loss: 1.5510 - val_accuracy: 0.3333
11/11 [==============================] - 1s 93ms/step
3/3 [==============================] - 0s 91ms/step
11/11 [==============================] - 1s 90ms/step
3/3 [==============================] - 0s 89ms/step
Model: "sequential_1225"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_5 (Embedding)     (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_5 (Spatia  (None, 200, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_5 (LSTM)               (None, 200)               173600    
                                                                 
 dense_3668 (Dense)          (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 15s 850ms/step - loss: 1.5579 - accuracy: 0.2786 - val_loss: 1.5383 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 12s 889ms/step - loss: 1.4996 - accuracy: 0.3359 - val_loss: 1.5488 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 13s 908ms/step - loss: 1.4731 - accuracy: 0.3397 - val_loss: 1.5506 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 13s 909ms/step - loss: 1.4780 - accuracy: 0.3282 - val_loss: 1.5444 - val_accuracy: 0.3333
11/11 [==============================] - 1s 91ms/step
3/3 [==============================] - 0s 91ms/step
11/11 [==============================] - 1s 92ms/step
3/3 [==============================] - 0s 91ms/step
Result of all runs:             model  train accuracy  test accuracy  train F1 score  \
0  Neural Network        0.335366       0.337349        0.168449   
1  Neural Network        0.335366       0.337349        0.168449   
2  Neural Network        0.335366       0.337349        0.168449   
3  Neural Network        0.335366       0.337349        0.168449   
4  Neural Network        0.335366       0.337349        0.168449   

   test F1 score  
0       0.170194  
1       0.170194  
2       0.170194  
3       0.170194  
4       0.170194  

LSTM_Model(X_train_cv_smote, X_test_cv, y_train_cv_smote, y_test_cv)

Model: "sequential_1226"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_6 (Embedding)     (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_6 (Spatia  (None, 200, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_6 (LSTM)               (None, 200)               173600    
                                                                 
 dense_3669 (Dense)          (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 21s 896ms/step - loss: 1.5784 - accuracy: 0.2114 - val_loss: 2.0641 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 19s 840ms/step - loss: 1.5223 - accuracy: 0.2614 - val_loss: 2.2440 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 18s 807ms/step - loss: 1.5254 - accuracy: 0.2500 - val_loss: 2.7502 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 18s 824ms/step - loss: 1.5250 - accuracy: 0.2500 - val_loss: 2.3848 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 91ms/step
3/3 [==============================] - 0s 90ms/step
18/18 [==============================] - 2s 136ms/step
3/3 [==============================] - 0s 134ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.274545	0.253012	0.159607	0.172921

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(5):
  result=LSTM_Model(X_train_cv_smote, X_test_cv, y_train_cv_smote, y_test_cv)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_1227"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_7 (Embedding)     (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_7 (Spatia  (None, 200, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_7 (LSTM)               (None, 200)               173600    
                                                                 
 dense_3670 (Dense)          (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 20s 831ms/step - loss: 1.5624 - accuracy: 0.2091 - val_loss: 1.9595 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 18s 839ms/step - loss: 1.5364 - accuracy: 0.2705 - val_loss: 2.1311 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 18s 821ms/step - loss: 1.5320 - accuracy: 0.2500 - val_loss: 2.5942 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 18s 804ms/step - loss: 1.5258 - accuracy: 0.2409 - val_loss: 2.4219 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 97ms/step
3/3 [==============================] - 0s 90ms/step
18/18 [==============================] - 2s 132ms/step
3/3 [==============================] - 0s 136ms/step
Model: "sequential_1228"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_8 (Embedding)     (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_8 (Spatia  (None, 200, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_8 (LSTM)               (None, 200)               173600    
                                                                 
 dense_3671 (Dense)          (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 22s 906ms/step - loss: 1.5703 - accuracy: 0.2364 - val_loss: 1.8933 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 19s 858ms/step - loss: 1.5466 - accuracy: 0.2750 - val_loss: 2.3889 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 18s 804ms/step - loss: 1.5230 - accuracy: 0.2682 - val_loss: 2.5352 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 18s 801ms/step - loss: 1.5219 - accuracy: 0.2682 - val_loss: 2.4791 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 94ms/step
3/3 [==============================] - 0s 96ms/step
18/18 [==============================] - 2s 120ms/step
3/3 [==============================] - 0s 139ms/step
Model: "sequential_1229"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_9 (Embedding)     (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_9 (Spatia  (None, 200, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_9 (LSTM)               (None, 200)               173600    
                                                                 
 dense_3672 (Dense)          (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 20s 825ms/step - loss: 1.5608 - accuracy: 0.2295 - val_loss: 2.3702 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 18s 815ms/step - loss: 1.5193 - accuracy: 0.2636 - val_loss: 2.0598 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 19s 883ms/step - loss: 1.5323 - accuracy: 0.2500 - val_loss: 2.7993 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 18s 812ms/step - loss: 1.5294 - accuracy: 0.2477 - val_loss: 2.3788 - val_accuracy: 0.0000e+00
Epoch 5/100
22/22 [==============================] - 18s 802ms/step - loss: 1.5251 - accuracy: 0.2591 - val_loss: 2.4772 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 122ms/step
3/3 [==============================] - 0s 138ms/step
18/18 [==============================] - 2s 125ms/step
3/3 [==============================] - 0s 94ms/step
Model: "sequential_1230"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_10 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_10 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_10 (LSTM)              (None, 200)               173600    
                                                                 
 dense_3673 (Dense)          (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 22s 900ms/step - loss: 1.5668 - accuracy: 0.2227 - val_loss: 1.9413 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 19s 886ms/step - loss: 1.5399 - accuracy: 0.2659 - val_loss: 2.1834 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 18s 809ms/step - loss: 1.5288 - accuracy: 0.2523 - val_loss: 2.6064 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 18s 808ms/step - loss: 1.5267 - accuracy: 0.2500 - val_loss: 2.3918 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 92ms/step
3/3 [==============================] - 0s 94ms/step
18/18 [==============================] - 2s 94ms/step
3/3 [==============================] - 0s 88ms/step
Model: "sequential_1231"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_11 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_11 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_11 (LSTM)              (None, 200)               173600    
                                                                 
 dense_3674 (Dense)          (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 21s 837ms/step - loss: 1.5717 - accuracy: 0.2227 - val_loss: 2.2419 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 18s 817ms/step - loss: 1.5142 - accuracy: 0.2841 - val_loss: 2.1966 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 18s 848ms/step - loss: 1.5252 - accuracy: 0.2568 - val_loss: 2.7989 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 19s 828ms/step - loss: 1.5284 - accuracy: 0.2523 - val_loss: 2.3139 - val_accuracy: 0.0000e+00
Epoch 5/100
22/22 [==============================] - 20s 902ms/step - loss: 1.5250 - accuracy: 0.2750 - val_loss: 2.4902 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 115ms/step
3/3 [==============================] - 0s 131ms/step
18/18 [==============================] - 2s 118ms/step
3/3 [==============================] - 0s 92ms/step
Result of all runs:             model  train accuracy  test accuracy  train F1 score  \
0  Neural Network        0.200000       0.228916        0.066667   
1  Neural Network        0.232727       0.144578        0.125578   
2  Neural Network        0.200000       0.228916        0.066667   
3  Neural Network        0.200000       0.337349        0.066667   
4  Neural Network        0.203636       0.132530        0.074175   

   test F1 score  
0       0.085282  
1       0.092572  
2       0.085282  
3       0.170194  
4       0.065459  

LSTM_Model(X_train_cvfull, X_test_cvfull, y_train_cvfull, y_test_cvfull)

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding (Embedding)       (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d (SpatialD  (None, 219, 16)          0         
 ropout1D)                                                       
                                                                 
 lstm (LSTM)                 (None, 200)               173600    
                                                                 
 dense (Dense)               (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 19s 990ms/step - loss: 1.5576 - accuracy: 0.2977 - val_loss: 1.5400 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 13s 942ms/step - loss: 1.5094 - accuracy: 0.3359 - val_loss: 1.5392 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 13s 944ms/step - loss: 1.4809 - accuracy: 0.2939 - val_loss: 1.5598 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 13s 948ms/step - loss: 1.4783 - accuracy: 0.2977 - val_loss: 1.5437 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 13s 944ms/step - loss: 1.4622 - accuracy: 0.3359 - val_loss: 1.5701 - val_accuracy: 0.3333
11/11 [==============================] - 2s 115ms/step
3/3 [==============================] - 0s 108ms/step
11/11 [==============================] - 2s 141ms/step
3/3 [==============================] - 1s 187ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.335366	0.337349	0.168449	0.170194

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(5):
  result=LSTM_Model(X_train_cvfull, X_test_cvfull, y_train_cvfull, y_test_cvfull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1 (Embedding)     (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_1 (Spatia  (None, 219, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_1 (LSTM)               (None, 200)               173600    
                                                                 
 dense_1 (Dense)             (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 18s 1s/step - loss: 1.5654 - accuracy: 0.2901 - val_loss: 1.5427 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 13s 944ms/step - loss: 1.5136 - accuracy: 0.3359 - val_loss: 1.5371 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 13s 943ms/step - loss: 1.4834 - accuracy: 0.3359 - val_loss: 1.5553 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 13s 945ms/step - loss: 1.4766 - accuracy: 0.3321 - val_loss: 1.5509 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 13s 941ms/step - loss: 1.4622 - accuracy: 0.3359 - val_loss: 1.5742 - val_accuracy: 0.3333
11/11 [==============================] - 2s 111ms/step
3/3 [==============================] - 0s 106ms/step
11/11 [==============================] - 1s 110ms/step
3/3 [==============================] - 0s 105ms/step
Model: "sequential_2"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_2 (Embedding)     (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_2 (Spatia  (None, 219, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_2 (LSTM)               (None, 200)               173600    
                                                                 
 dense_2 (Dense)             (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 17s 924ms/step - loss: 1.5609 - accuracy: 0.2824 - val_loss: 1.5386 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 13s 888ms/step - loss: 1.5081 - accuracy: 0.3359 - val_loss: 1.5340 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 13s 922ms/step - loss: 1.4794 - accuracy: 0.3282 - val_loss: 1.5513 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 13s 946ms/step - loss: 1.4787 - accuracy: 0.3282 - val_loss: 1.5467 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 13s 939ms/step - loss: 1.4628 - accuracy: 0.3359 - val_loss: 1.5756 - val_accuracy: 0.3333
11/11 [==============================] - 1s 110ms/step
3/3 [==============================] - 0s 103ms/step
11/11 [==============================] - 1s 110ms/step
3/3 [==============================] - 0s 109ms/step
Model: "sequential_3"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_3 (Embedding)     (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_3 (Spatia  (None, 219, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_3 (LSTM)               (None, 200)               173600    
                                                                 
 dense_3 (Dense)             (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 17s 988ms/step - loss: 1.5532 - accuracy: 0.2824 - val_loss: 1.5547 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 13s 950ms/step - loss: 1.5233 - accuracy: 0.3359 - val_loss: 1.5406 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 13s 962ms/step - loss: 1.4817 - accuracy: 0.3359 - val_loss: 1.5694 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 13s 959ms/step - loss: 1.4790 - accuracy: 0.3359 - val_loss: 1.5417 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 13s 956ms/step - loss: 1.4637 - accuracy: 0.3359 - val_loss: 1.5677 - val_accuracy: 0.3333
11/11 [==============================] - 2s 151ms/step
3/3 [==============================] - 1s 180ms/step
11/11 [==============================] - 1s 113ms/step
3/3 [==============================] - 0s 111ms/step
Model: "sequential_4"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_4 (Embedding)     (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_4 (Spatia  (None, 219, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_4 (LSTM)               (None, 200)               173600    
                                                                 
 dense_4 (Dense)             (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 17s 990ms/step - loss: 1.5506 - accuracy: 0.2595 - val_loss: 1.5434 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 13s 952ms/step - loss: 1.5148 - accuracy: 0.3359 - val_loss: 1.5322 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 13s 937ms/step - loss: 1.4774 - accuracy: 0.3359 - val_loss: 1.5639 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 13s 940ms/step - loss: 1.4730 - accuracy: 0.3282 - val_loss: 1.5491 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 14s 1s/step - loss: 1.4604 - accuracy: 0.3359 - val_loss: 1.5724 - val_accuracy: 0.3333
11/11 [==============================] - 1s 110ms/step
3/3 [==============================] - 0s 154ms/step
11/11 [==============================] - 2s 179ms/step
3/3 [==============================] - 1s 178ms/step
Model: "sequential_5"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_5 (Embedding)     (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_5 (Spatia  (None, 219, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_5 (LSTM)               (None, 200)               173600    
                                                                 
 dense_5 (Dense)             (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 17s 977ms/step - loss: 1.5617 - accuracy: 0.2939 - val_loss: 1.5408 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 13s 943ms/step - loss: 1.5139 - accuracy: 0.3359 - val_loss: 1.5372 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 13s 941ms/step - loss: 1.4828 - accuracy: 0.3359 - val_loss: 1.5538 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 13s 940ms/step - loss: 1.4741 - accuracy: 0.3092 - val_loss: 1.5475 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 13s 943ms/step - loss: 1.4600 - accuracy: 0.3359 - val_loss: 1.5728 - val_accuracy: 0.3333
11/11 [==============================] - 2s 110ms/step
3/3 [==============================] - 0s 113ms/step
11/11 [==============================] - 1s 111ms/step
3/3 [==============================] - 0s 111ms/step
Result of all runs:             model  train accuracy  test accuracy  train F1 score  \
0  Neural Network        0.335366       0.337349        0.168449   
1  Neural Network        0.335366       0.337349        0.168449   
2  Neural Network        0.335366       0.337349        0.168449   
3  Neural Network        0.335366       0.337349        0.168449   
4  Neural Network        0.335366       0.337349        0.168449   

   test F1 score  
0       0.170194  
1       0.170194  
2       0.170194  
3       0.170194  
4       0.170194  

LSTM_Model(X_train_cvfull_smote, X_test_cvfull, y_train_cvfull_smote, y_test_cvfull)

Model: "sequential_6"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_6 (Embedding)     (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_6 (Spatia  (None, 219, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_6 (LSTM)               (None, 200)               173600    
                                                                 
 dense_6 (Dense)             (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 22s 860ms/step - loss: 1.5738 - accuracy: 0.2295 - val_loss: 2.0780 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 19s 889ms/step - loss: 1.5221 - accuracy: 0.2705 - val_loss: 2.1755 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 22s 967ms/step - loss: 1.5283 - accuracy: 0.2500 - val_loss: 2.6966 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 22s 999ms/step - loss: 1.5265 - accuracy: 0.2432 - val_loss: 2.4087 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 112ms/step
3/3 [==============================] - 0s 108ms/step
18/18 [==============================] - 2s 113ms/step
3/3 [==============================] - 0s 105ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.2	0.108434	0.066667	0.021215

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(5):
  result=LSTM_Model(X_train_cvfull_smote, X_test_cvfull, y_train_cvfull_smote, y_test_cvfull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_7"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_7 (Embedding)     (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_7 (Spatia  (None, 219, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_7 (LSTM)               (None, 200)               173600    
                                                                 
 dense_7 (Dense)             (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 39s 2s/step - loss: 1.5653 - accuracy: 0.2182 - val_loss: 2.4294 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 30s 1s/step - loss: 1.5125 - accuracy: 0.2818 - val_loss: 2.2171 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 36s 2s/step - loss: 1.5254 - accuracy: 0.2500 - val_loss: 2.7950 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 19s 845ms/step - loss: 1.5269 - accuracy: 0.2364 - val_loss: 2.3299 - val_accuracy: 0.0000e+00
Epoch 5/100
22/22 [==============================] - 22s 1s/step - loss: 1.5257 - accuracy: 0.2455 - val_loss: 2.4424 - val_accuracy: 0.0000e+00
18/18 [==============================] - 5s 242ms/step
3/3 [==============================] - 0s 129ms/step
18/18 [==============================] - 4s 219ms/step
3/3 [==============================] - 1s 202ms/step
Model: "sequential_8"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_8 (Embedding)     (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_8 (Spatia  (None, 219, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_8 (LSTM)               (None, 200)               173600    
                                                                 
 dense_8 (Dense)             (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 49s 2s/step - loss: 1.5783 - accuracy: 0.2205 - val_loss: 1.9577 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 39s 2s/step - loss: 1.5301 - accuracy: 0.2636 - val_loss: 2.1338 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 36s 2s/step - loss: 1.5307 - accuracy: 0.2500 - val_loss: 2.5970 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 41s 2s/step - loss: 1.5254 - accuracy: 0.2477 - val_loss: 2.4621 - val_accuracy: 0.0000e+00
18/18 [==============================] - 4s 209ms/step
3/3 [==============================] - 1s 281ms/step
18/18 [==============================] - 7s 368ms/step
3/3 [==============================] - 1s 209ms/step
Model: "sequential_9"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_9 (Embedding)     (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_9 (Spatia  (None, 219, 16)          0         
 lDropout1D)                                                     
                                                                 
 lstm_9 (LSTM)               (None, 200)               173600    
                                                                 
 dense_9 (Dense)             (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 45s 2s/step - loss: 1.5615 - accuracy: 0.2295 - val_loss: 2.0839 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 35s 2s/step - loss: 1.5286 - accuracy: 0.2636 - val_loss: 2.0618 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 18s 845ms/step - loss: 1.5358 - accuracy: 0.2341 - val_loss: 2.6341 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 20s 922ms/step - loss: 1.5270 - accuracy: 0.2455 - val_loss: 2.4394 - val_accuracy: 0.0000e+00
Epoch 5/100
22/22 [==============================] - 22s 1s/step - loss: 1.5264 - accuracy: 0.2682 - val_loss: 2.4560 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 111ms/step
3/3 [==============================] - 0s 109ms/step
18/18 [==============================] - 2s 113ms/step
3/3 [==============================] - 0s 108ms/step
Model: "sequential_10"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_10 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_10 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_10 (LSTM)              (None, 200)               173600    
                                                                 
 dense_10 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 23s 974ms/step - loss: 1.5667 - accuracy: 0.2091 - val_loss: 2.0588 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 19s 889ms/step - loss: 1.5301 - accuracy: 0.2636 - val_loss: 2.0634 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 22s 999ms/step - loss: 1.5363 - accuracy: 0.2591 - val_loss: 2.4660 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 18s 834ms/step - loss: 1.5260 - accuracy: 0.2568 - val_loss: 2.4947 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 112ms/step
3/3 [==============================] - 0s 103ms/step
18/18 [==============================] - 2s 124ms/step
3/3 [==============================] - 1s 174ms/step
Model: "sequential_11"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_11 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_11 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_11 (LSTM)              (None, 200)               173600    
                                                                 
 dense_11 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 21s 855ms/step - loss: 1.5626 - accuracy: 0.2205 - val_loss: 2.4031 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 19s 886ms/step - loss: 1.5160 - accuracy: 0.2682 - val_loss: 2.1680 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 19s 835ms/step - loss: 1.5272 - accuracy: 0.2500 - val_loss: 2.8518 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 18s 841ms/step - loss: 1.5281 - accuracy: 0.2409 - val_loss: 2.2998 - val_accuracy: 0.0000e+00
Epoch 5/100
22/22 [==============================] - 20s 925ms/step - loss: 1.5259 - accuracy: 0.2591 - val_loss: 2.3901 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 111ms/step
3/3 [==============================] - 0s 105ms/step
18/18 [==============================] - 2s 111ms/step
3/3 [==============================] - 0s 104ms/step
Result of all runs:             model  train accuracy  test accuracy  train F1 score  \
0  Neural Network        0.247273       0.204819        0.142357   
1  Neural Network        0.200000       0.108434        0.066667   
2  Neural Network        0.210909       0.168675        0.091154   
3  Neural Network        0.201818       0.108434        0.070371   
4  Neural Network        0.209091       0.144578        0.090549   

   test F1 score  
0       0.102380  
1       0.021215  
2       0.102739  
3       0.021215  
4       0.077635  

Observations-

TFIDF dataset-

LSTM_Model(X_train_tfidf, X_test_tfidf, y_train_tfidf, y_test_tfidf)

Model: "sequential_12"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_12 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_12 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_12 (LSTM)              (None, 200)               173600    
                                                                 
 dense_12 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 17s 996ms/step - loss: 1.5757 - accuracy: 0.2672 - val_loss: 1.5498 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 14s 962ms/step - loss: 1.5349 - accuracy: 0.3397 - val_loss: 1.5556 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 12s 898ms/step - loss: 1.4964 - accuracy: 0.2557 - val_loss: 1.5716 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 12s 889ms/step - loss: 1.4897 - accuracy: 0.3359 - val_loss: 1.5390 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 11s 808ms/step - loss: 1.4719 - accuracy: 0.3359 - val_loss: 1.5461 - val_accuracy: 0.3333
Epoch 6/100
14/14 [==============================] - 12s 786ms/step - loss: 1.4615 - accuracy: 0.2824 - val_loss: 1.5882 - val_accuracy: 0.3333
Epoch 7/100
14/14 [==============================] - 12s 870ms/step - loss: 1.4697 - accuracy: 0.3359 - val_loss: 1.5475 - val_accuracy: 0.3333
11/11 [==============================] - 1s 101ms/step
3/3 [==============================] - 0s 98ms/step
11/11 [==============================] - 1s 103ms/step
3/3 [==============================] - 0s 98ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.335366	0.337349	0.168449	0.170194

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model(X_train_tfidf, X_test_tfidf, y_train_tfidf, y_test_tfidf)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_13"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_13 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_13 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_13 (LSTM)              (None, 200)               173600    
                                                                 
 dense_13 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 31s 2s/step - loss: 1.5622 - accuracy: 0.2748 - val_loss: 1.5393 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 23s 2s/step - loss: 1.5064 - accuracy: 0.3359 - val_loss: 1.5343 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 21s 2s/step - loss: 1.4729 - accuracy: 0.3206 - val_loss: 1.5512 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 25s 2s/step - loss: 1.4808 - accuracy: 0.3397 - val_loss: 1.5418 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 24s 2s/step - loss: 1.4633 - accuracy: 0.3359 - val_loss: 1.5674 - val_accuracy: 0.3333
11/11 [==============================] - 3s 221ms/step
3/3 [==============================] - 1s 184ms/step
11/11 [==============================] - 2s 185ms/step
3/3 [==============================] - 1s 168ms/step
Model: "sequential_14"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_14 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_14 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_14 (LSTM)              (None, 200)               173600    
                                                                 
 dense_14 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 33s 2s/step - loss: 1.5714 - accuracy: 0.2634 - val_loss: 1.5420 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 26s 2s/step - loss: 1.5269 - accuracy: 0.3359 - val_loss: 1.5510 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 28s 2s/step - loss: 1.4954 - accuracy: 0.2672 - val_loss: 1.5727 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 27s 2s/step - loss: 1.4943 - accuracy: 0.3282 - val_loss: 1.5387 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 21s 2s/step - loss: 1.4727 - accuracy: 0.3359 - val_loss: 1.5445 - val_accuracy: 0.3333
Epoch 6/100
14/14 [==============================] - 16s 1s/step - loss: 1.4630 - accuracy: 0.3168 - val_loss: 1.5863 - val_accuracy: 0.3333
Epoch 7/100
14/14 [==============================] - 12s 829ms/step - loss: 1.4676 - accuracy: 0.3359 - val_loss: 1.5493 - val_accuracy: 0.3333
11/11 [==============================] - 2s 116ms/step
3/3 [==============================] - 0s 101ms/step
11/11 [==============================] - 1s 103ms/step
3/3 [==============================] - 0s 95ms/step
Model: "sequential_15"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_15 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_15 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_15 (LSTM)              (None, 200)               173600    
                                                                 
 dense_15 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 16s 906ms/step - loss: 1.5428 - accuracy: 0.2748 - val_loss: 1.5475 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 12s 883ms/step - loss: 1.5141 - accuracy: 0.3359 - val_loss: 1.5346 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 13s 967ms/step - loss: 1.4814 - accuracy: 0.3359 - val_loss: 1.5458 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 12s 874ms/step - loss: 1.4732 - accuracy: 0.3359 - val_loss: 1.5455 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 14s 1s/step - loss: 1.4625 - accuracy: 0.3359 - val_loss: 1.5715 - val_accuracy: 0.3333
11/11 [==============================] - 1s 104ms/step
3/3 [==============================] - 0s 96ms/step
11/11 [==============================] - 2s 170ms/step
3/3 [==============================] - 0s 90ms/step
Result of all runs:             model  train accuracy  test accuracy  train F1 score  \
0  Neural Network        0.335366       0.337349        0.168449   
1  Neural Network        0.335366       0.337349        0.168449   
2  Neural Network        0.335366       0.337349        0.168449   

   test F1 score  
0       0.170194  
1       0.170194  
2       0.170194  

LSTM_Model(X_train_tfidf_smote, X_test_tfidf, y_train_tfidf_smote, y_test_tfidf)

Model: "sequential_16"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_16 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_16 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_16 (LSTM)              (None, 200)               173600    
                                                                 
 dense_16 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 20s 835ms/step - loss: 1.5687 - accuracy: 0.2045 - val_loss: 1.9298 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 18s 829ms/step - loss: 1.5429 - accuracy: 0.2705 - val_loss: 2.1725 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 19s 828ms/step - loss: 1.5311 - accuracy: 0.2159 - val_loss: 2.5459 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 17s 779ms/step - loss: 1.5266 - accuracy: 0.2477 - val_loss: 2.4628 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 105ms/step
3/3 [==============================] - 0s 100ms/step
18/18 [==============================] - 2s 120ms/step
3/3 [==============================] - 1s 157ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.2	0.337349	0.066667	0.170194

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model(X_train_tfidf_smote, X_test_tfidf, y_train_tfidf_smote, y_test_tfidf)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_17"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_17 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_17 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_17 (LSTM)              (None, 200)               173600    
                                                                 
 dense_17 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 20s 810ms/step - loss: 1.5652 - accuracy: 0.2455 - val_loss: 2.5897 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 19s 859ms/step - loss: 1.5127 - accuracy: 0.2659 - val_loss: 2.1476 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 20s 897ms/step - loss: 1.5283 - accuracy: 0.2500 - val_loss: 2.8259 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 17s 785ms/step - loss: 1.5311 - accuracy: 0.2500 - val_loss: 2.2689 - val_accuracy: 0.0000e+00
Epoch 5/100
22/22 [==============================] - 18s 840ms/step - loss: 1.5288 - accuracy: 0.2023 - val_loss: 2.4044 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 118ms/step
3/3 [==============================] - 0s 97ms/step
18/18 [==============================] - 2s 103ms/step
3/3 [==============================] - 0s 98ms/step
Model: "sequential_18"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_18 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_18 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_18 (LSTM)              (None, 200)               173600    
                                                                 
 dense_18 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 20s 835ms/step - loss: 1.5621 - accuracy: 0.2159 - val_loss: 2.0407 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 18s 778ms/step - loss: 1.5354 - accuracy: 0.2568 - val_loss: 2.1319 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 17s 776ms/step - loss: 1.5328 - accuracy: 0.2500 - val_loss: 2.5036 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 17s 770ms/step - loss: 1.5265 - accuracy: 0.2386 - val_loss: 2.4562 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 163ms/step
3/3 [==============================] - 1s 163ms/step
18/18 [==============================] - 2s 103ms/step
3/3 [==============================] - 0s 93ms/step
Model: "sequential_19"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_19 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_19 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_19 (LSTM)              (None, 200)               173600    
                                                                 
 dense_19 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 21s 815ms/step - loss: 1.5778 - accuracy: 0.2273 - val_loss: 1.9212 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 20s 887ms/step - loss: 1.5447 - accuracy: 0.2659 - val_loss: 2.3197 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 17s 774ms/step - loss: 1.5253 - accuracy: 0.2477 - val_loss: 2.6127 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 18s 811ms/step - loss: 1.5270 - accuracy: 0.2295 - val_loss: 2.4342 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 119ms/step
3/3 [==============================] - 1s 166ms/step
18/18 [==============================] - 3s 147ms/step
3/3 [==============================] - 0s 94ms/step
Result of all runs:             model  train accuracy  test accuracy  train F1 score  \
0  Neural Network             0.2       0.337349        0.066667   
1  Neural Network             0.2       0.337349        0.066667   
2  Neural Network             0.2       0.228916        0.066667   

   test F1 score  
0       0.170194  
1       0.170194  
2       0.085282  

LSTM_Model(X_train_tfidffull, X_test_tfidffull, y_train_tfidffull, y_test_tfidffull)

Model: "sequential_20"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_20 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_20 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_20 (LSTM)              (None, 200)               173600    
                                                                 
 dense_20 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 17s 982ms/step - loss: 1.5590 - accuracy: 0.2634 - val_loss: 1.5376 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 13s 955ms/step - loss: 1.4952 - accuracy: 0.3359 - val_loss: 1.5388 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 13s 967ms/step - loss: 1.4721 - accuracy: 0.3359 - val_loss: 1.5503 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 13s 953ms/step - loss: 1.4787 - accuracy: 0.3015 - val_loss: 1.5434 - val_accuracy: 0.3333
11/11 [==============================] - 2s 177ms/step
3/3 [==============================] - 0s 105ms/step
11/11 [==============================] - 1s 111ms/step
3/3 [==============================] - 0s 103ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.335366	0.337349	0.168449	0.170194

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model(X_train_tfidffull, X_test_tfidffull, y_train_tfidffull, y_test_tfidffull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_21"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_21 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_21 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_21 (LSTM)              (None, 200)               173600    
                                                                 
 dense_21 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 19s 1s/step - loss: 1.5633 - accuracy: 0.2710 - val_loss: 1.5400 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 13s 948ms/step - loss: 1.5223 - accuracy: 0.3359 - val_loss: 1.5441 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 15s 1s/step - loss: 1.4929 - accuracy: 0.2786 - val_loss: 1.5526 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 22s 1s/step - loss: 1.4866 - accuracy: 0.3015 - val_loss: 1.5413 - val_accuracy: 0.3333
11/11 [==============================] - 2s 147ms/step
3/3 [==============================] - 0s 145ms/step
11/11 [==============================] - 2s 180ms/step
3/3 [==============================] - 1s 553ms/step
Model: "sequential_22"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_22 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_22 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_22 (LSTM)              (None, 200)               173600    
                                                                 
 dense_22 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 28s 2s/step - loss: 1.5779 - accuracy: 0.2634 - val_loss: 1.5459 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 16s 1s/step - loss: 1.5377 - accuracy: 0.3359 - val_loss: 1.5563 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 13s 947ms/step - loss: 1.5072 - accuracy: 0.2939 - val_loss: 1.5552 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 13s 947ms/step - loss: 1.4921 - accuracy: 0.3359 - val_loss: 1.5425 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 17s 1s/step - loss: 1.4699 - accuracy: 0.3359 - val_loss: 1.5537 - val_accuracy: 0.3333
Epoch 6/100
14/14 [==============================] - 24s 2s/step - loss: 1.4629 - accuracy: 0.3435 - val_loss: 1.5838 - val_accuracy: 0.3333
Epoch 7/100
14/14 [==============================] - 17s 1s/step - loss: 1.4670 - accuracy: 0.3359 - val_loss: 1.5509 - val_accuracy: 0.3333
11/11 [==============================] - 3s 196ms/step
3/3 [==============================] - 1s 183ms/step
11/11 [==============================] - 3s 234ms/step
3/3 [==============================] - 1s 183ms/step
Model: "sequential_23"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_23 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_23 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_23 (LSTM)              (None, 200)               173600    
                                                                 
 dense_23 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 24s 1s/step - loss: 1.5757 - accuracy: 0.2824 - val_loss: 1.5433 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 19s 1s/step - loss: 1.5299 - accuracy: 0.3359 - val_loss: 1.5496 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 25s 2s/step - loss: 1.4948 - accuracy: 0.3359 - val_loss: 1.5755 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 18s 1s/step - loss: 1.4927 - accuracy: 0.3053 - val_loss: 1.5418 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 21s 2s/step - loss: 1.4746 - accuracy: 0.3359 - val_loss: 1.5427 - val_accuracy: 0.3333
Epoch 6/100
14/14 [==============================] - 18s 1s/step - loss: 1.4615 - accuracy: 0.3359 - val_loss: 1.5824 - val_accuracy: 0.3333
Epoch 7/100
14/14 [==============================] - 18s 1s/step - loss: 1.4710 - accuracy: 0.3359 - val_loss: 1.5490 - val_accuracy: 0.3333
11/11 [==============================] - 3s 215ms/step
3/3 [==============================] - 1s 260ms/step
11/11 [==============================] - 2s 216ms/step
3/3 [==============================] - 1s 180ms/step
Result of all runs:             model  train accuracy  test accuracy  train F1 score  \
0  Neural Network        0.335366       0.337349        0.168449   
1  Neural Network        0.335366       0.337349        0.168449   
2  Neural Network        0.335366       0.337349        0.168449   

   test F1 score  
0       0.170194  
1       0.170194  
2       0.170194  

LSTM_Model(X_train_tfidffull_smote, X_test_tfidffull, y_train_tfidffull_smote, y_test_tfidffull)

Model: "sequential_24"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_24 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_24 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_24 (LSTM)              (None, 200)               173600    
                                                                 
 dense_24 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 40s 2s/step - loss: 1.5695 - accuracy: 0.2114 - val_loss: 2.3985 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 21s 966ms/step - loss: 1.5183 - accuracy: 0.2614 - val_loss: 2.0846 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 20s 917ms/step - loss: 1.5333 - accuracy: 0.2545 - val_loss: 2.7344 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 18s 841ms/step - loss: 1.5300 - accuracy: 0.2409 - val_loss: 2.3910 - val_accuracy: 0.0000e+00
Epoch 5/100
22/22 [==============================] - 22s 1s/step - loss: 1.5267 - accuracy: 0.2636 - val_loss: 2.4663 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 112ms/step
3/3 [==============================] - 0s 105ms/step
18/18 [==============================] - 2s 111ms/step
3/3 [==============================] - 0s 105ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.2	0.228916	0.066667	0.085282

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model(X_train_tfidffull_smote, X_test_tfidffull, y_train_tfidffull_smote, y_test_tfidffull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_25"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_25 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_25 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_25 (LSTM)              (None, 200)               173600    
                                                                 
 dense_25 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 24s 879ms/step - loss: 1.5638 - accuracy: 0.2091 - val_loss: 2.1825 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 18s 842ms/step - loss: 1.5235 - accuracy: 0.2750 - val_loss: 2.0534 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 20s 922ms/step - loss: 1.5336 - accuracy: 0.2455 - val_loss: 2.6876 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 18s 836ms/step - loss: 1.5290 - accuracy: 0.2409 - val_loss: 2.3821 - val_accuracy: 0.0000e+00
Epoch 5/100
22/22 [==============================] - 20s 901ms/step - loss: 1.5271 - accuracy: 0.2295 - val_loss: 2.4382 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 153ms/step
3/3 [==============================] - 0s 104ms/step
18/18 [==============================] - 2s 111ms/step
3/3 [==============================] - 0s 110ms/step
Model: "sequential_26"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_26 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_26 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_26 (LSTM)              (None, 200)               173600    
                                                                 
 dense_26 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 23s 934ms/step - loss: 1.5701 - accuracy: 0.2364 - val_loss: 2.3831 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 18s 837ms/step - loss: 1.5141 - accuracy: 0.2727 - val_loss: 2.1627 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 19s 853ms/step - loss: 1.5269 - accuracy: 0.2500 - val_loss: 2.7957 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 20s 873ms/step - loss: 1.5319 - accuracy: 0.2205 - val_loss: 2.2972 - val_accuracy: 0.0000e+00
Epoch 5/100
22/22 [==============================] - 18s 833ms/step - loss: 1.5282 - accuracy: 0.2182 - val_loss: 2.4334 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 111ms/step
3/3 [==============================] - 0s 106ms/step
18/18 [==============================] - 3s 177ms/step
3/3 [==============================] - 1s 150ms/step
Model: "sequential_27"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_27 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_27 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_27 (LSTM)              (None, 200)               173600    
                                                                 
 dense_27 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 21s 854ms/step - loss: 1.5702 - accuracy: 0.2068 - val_loss: 2.1585 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 20s 918ms/step - loss: 1.5187 - accuracy: 0.2841 - val_loss: 2.2189 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 18s 832ms/step - loss: 1.5283 - accuracy: 0.2523 - val_loss: 2.7024 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 19s 870ms/step - loss: 1.5300 - accuracy: 0.2341 - val_loss: 2.3409 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 111ms/step
3/3 [==============================] - 0s 102ms/step
18/18 [==============================] - 2s 124ms/step
3/3 [==============================] - 1s 177ms/step
Result of all runs:             model  train accuracy  test accuracy  train F1 score  \
0  Neural Network             0.2       0.108434        0.066667   
1  Neural Network             0.2       0.108434        0.066667   
2  Neural Network             0.2       0.228916        0.066667   

   test F1 score  
0       0.021215  
1       0.021215  
2       0.085282  

Observations-

Word2Vec Dataset-

LSTM_Model(X_train_wv, X_test_wv, y_train_wv, y_test_wv)

Model: "sequential_28"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_28 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_28 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_28 (LSTM)              (None, 200)               173600    
                                                                 
 dense_28 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 16s 901ms/step - loss: 1.5844 - accuracy: 0.2748 - val_loss: 1.5507 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 12s 864ms/step - loss: 1.5419 - accuracy: 0.3168 - val_loss: 1.5624 - val_accuracy: 0.2273
Epoch 3/100
14/14 [==============================] - 12s 841ms/step - loss: 1.5128 - accuracy: 0.2290 - val_loss: 1.5591 - val_accuracy: 0.2273
Epoch 4/100
14/14 [==============================] - 11s 735ms/step - loss: 1.4937 - accuracy: 0.3168 - val_loss: 1.5405 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 12s 818ms/step - loss: 1.4686 - accuracy: 0.3359 - val_loss: 1.5552 - val_accuracy: 0.3333
Epoch 6/100
14/14 [==============================] - 13s 910ms/step - loss: 1.4681 - accuracy: 0.2748 - val_loss: 1.5789 - val_accuracy: 0.3333
Epoch 7/100
14/14 [==============================] - 13s 889ms/step - loss: 1.4663 - accuracy: 0.3397 - val_loss: 1.5502 - val_accuracy: 0.3333
11/11 [==============================] - 1s 100ms/step
3/3 [==============================] - 0s 94ms/step
11/11 [==============================] - 1s 99ms/step
3/3 [==============================] - 0s 93ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.335366	0.337349	0.168449	0.170194

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model(X_train_wv, X_test_wv, y_train_wv, y_test_wv)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_29"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_29 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_29 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_29 (LSTM)              (None, 200)               173600    
                                                                 
 dense_29 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 15s 772ms/step - loss: 1.5585 - accuracy: 0.2595 - val_loss: 1.5355 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 12s 803ms/step - loss: 1.5022 - accuracy: 0.3359 - val_loss: 1.5366 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 12s 876ms/step - loss: 1.4744 - accuracy: 0.3321 - val_loss: 1.5469 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 12s 867ms/step - loss: 1.4804 - accuracy: 0.3130 - val_loss: 1.5397 - val_accuracy: 0.3333
11/11 [==============================] - 1s 101ms/step
3/3 [==============================] - 0s 100ms/step
11/11 [==============================] - 1s 101ms/step
3/3 [==============================] - 0s 95ms/step
Model: "sequential_30"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_30 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_30 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_30 (LSTM)              (None, 200)               173600    
                                                                 
 dense_30 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 17s 916ms/step - loss: 1.5757 - accuracy: 0.2863 - val_loss: 1.5433 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 12s 809ms/step - loss: 1.5348 - accuracy: 0.3359 - val_loss: 1.5539 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 12s 865ms/step - loss: 1.5001 - accuracy: 0.2977 - val_loss: 1.5619 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 12s 867ms/step - loss: 1.4911 - accuracy: 0.3092 - val_loss: 1.5406 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 12s 870ms/step - loss: 1.4710 - accuracy: 0.3359 - val_loss: 1.5503 - val_accuracy: 0.3333
Epoch 6/100
14/14 [==============================] - 12s 869ms/step - loss: 1.4644 - accuracy: 0.3282 - val_loss: 1.5855 - val_accuracy: 0.3333
Epoch 7/100
14/14 [==============================] - 12s 858ms/step - loss: 1.4676 - accuracy: 0.3359 - val_loss: 1.5499 - val_accuracy: 0.3333
11/11 [==============================] - 2s 123ms/step
3/3 [==============================] - 1s 160ms/step
11/11 [==============================] - 2s 169ms/step
3/3 [==============================] - 0s 123ms/step
Model: "sequential_31"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_31 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_31 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_31 (LSTM)              (None, 200)               173600    
                                                                 
 dense_31 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 15s 895ms/step - loss: 1.5635 - accuracy: 0.2557 - val_loss: 1.5389 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 14s 1s/step - loss: 1.5094 - accuracy: 0.3359 - val_loss: 1.5363 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 11s 799ms/step - loss: 1.4773 - accuracy: 0.3206 - val_loss: 1.5537 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 11s 759ms/step - loss: 1.4792 - accuracy: 0.3359 - val_loss: 1.5432 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 12s 868ms/step - loss: 1.4642 - accuracy: 0.3359 - val_loss: 1.5726 - val_accuracy: 0.3333
11/11 [==============================] - 1s 101ms/step
3/3 [==============================] - 0s 95ms/step
11/11 [==============================] - 1s 104ms/step
3/3 [==============================] - 0s 104ms/step
Result of all runs:             model  train accuracy  test accuracy  train F1 score  \
0  Neural Network        0.335366       0.337349        0.168449   
1  Neural Network        0.335366       0.337349        0.168449   
2  Neural Network        0.335366       0.337349        0.168449   

   test F1 score  
0       0.170194  
1       0.170194  
2       0.170194  

LSTM_Model(X_train_wv_smote, X_test_wv, y_train_wv_smote, y_test_wv)

Model: "sequential_32"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_32 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_32 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_32 (LSTM)              (None, 200)               173600    
                                                                 
 dense_32 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 21s 800ms/step - loss: 1.5656 - accuracy: 0.2114 - val_loss: 2.0051 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 17s 774ms/step - loss: 1.5337 - accuracy: 0.2591 - val_loss: 2.0794 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 18s 804ms/step - loss: 1.5324 - accuracy: 0.2341 - val_loss: 2.7204 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 18s 785ms/step - loss: 1.5286 - accuracy: 0.2523 - val_loss: 2.3694 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 102ms/step
3/3 [==============================] - 0s 95ms/step
18/18 [==============================] - 2s 103ms/step
3/3 [==============================] - 0s 97ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.2	0.108434	0.066667	0.021215

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model(X_train_wv_smote, X_test_wv, y_train_wv_smote, y_test_wv)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_33"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_33 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_33 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_33 (LSTM)              (None, 200)               173600    
                                                                 
 dense_33 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 23s 890ms/step - loss: 1.5654 - accuracy: 0.2159 - val_loss: 2.2790 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 17s 768ms/step - loss: 1.5176 - accuracy: 0.2682 - val_loss: 2.1741 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 17s 794ms/step - loss: 1.5300 - accuracy: 0.2500 - val_loss: 2.7272 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 18s 784ms/step - loss: 1.5274 - accuracy: 0.2227 - val_loss: 2.3628 - val_accuracy: 0.0000e+00
Epoch 5/100
22/22 [==============================] - 25s 1s/step - loss: 1.5280 - accuracy: 0.2273 - val_loss: 2.4388 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 124ms/step
3/3 [==============================] - 1s 159ms/step
18/18 [==============================] - 2s 127ms/step
3/3 [==============================] - 0s 102ms/step
Model: "sequential_34"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_34 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_34 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_34 (LSTM)              (None, 200)               173600    
                                                                 
 dense_34 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 24s 1s/step - loss: 1.5683 - accuracy: 0.2250 - val_loss: 1.9697 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 38s 2s/step - loss: 1.5419 - accuracy: 0.2591 - val_loss: 2.2593 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 31s 1s/step - loss: 1.5296 - accuracy: 0.2500 - val_loss: 2.5681 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 29s 1s/step - loss: 1.5273 - accuracy: 0.2523 - val_loss: 2.4236 - val_accuracy: 0.0000e+00
18/18 [==============================] - 4s 172ms/step
3/3 [==============================] - 1s 203ms/step
18/18 [==============================] - 3s 168ms/step
3/3 [==============================] - 1s 177ms/step
Model: "sequential_35"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_35 (Embedding)    (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_35 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_35 (LSTM)              (None, 200)               173600    
                                                                 
 dense_35 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 45s 2s/step - loss: 1.5814 - accuracy: 0.2273 - val_loss: 2.0859 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 33s 1s/step - loss: 1.5233 - accuracy: 0.2659 - val_loss: 2.1618 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 33s 1s/step - loss: 1.5304 - accuracy: 0.2500 - val_loss: 2.6756 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 34s 2s/step - loss: 1.5277 - accuracy: 0.2227 - val_loss: 2.3892 - val_accuracy: 0.0000e+00
18/18 [==============================] - 4s 203ms/step
3/3 [==============================] - 1s 181ms/step
18/18 [==============================] - 3s 185ms/step
3/3 [==============================] - 1s 361ms/step
Result of all runs:             model  train accuracy  test accuracy  train F1 score  \
0  Neural Network             0.2       0.108434        0.066667   
1  Neural Network             0.2       0.337349        0.066667   
2  Neural Network             0.2       0.228916        0.066667   

   test F1 score  
0       0.021215  
1       0.170194  
2       0.085282  

LSTM_Model(X_train_wvfull, X_test_wvfull, y_train_wvfull, y_test_wvfull)

Model: "sequential_36"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_36 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_36 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_36 (LSTM)              (None, 200)               173600    
                                                                 
 dense_36 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 33s 2s/step - loss: 1.5557 - accuracy: 0.2748 - val_loss: 1.5476 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 28s 2s/step - loss: 1.5158 - accuracy: 0.3359 - val_loss: 1.5340 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 26s 2s/step - loss: 1.4758 - accuracy: 0.3359 - val_loss: 1.5629 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 22s 2s/step - loss: 1.4797 - accuracy: 0.3359 - val_loss: 1.5440 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 31s 2s/step - loss: 1.4629 - accuracy: 0.3359 - val_loss: 1.5708 - val_accuracy: 0.3333
11/11 [==============================] - 3s 265ms/step
3/3 [==============================] - 1s 162ms/step
11/11 [==============================] - 2s 183ms/step
3/3 [==============================] - 1s 173ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.335366	0.337349	0.168449	0.170194

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model(X_train_wvfull, X_test_wvfull, y_train_wvfull, y_test_wvfull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_37"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_37 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_37 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_37 (LSTM)              (None, 200)               173600    
                                                                 
 dense_37 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 19s 947ms/step - loss: 1.5571 - accuracy: 0.2863 - val_loss: 1.5372 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 13s 884ms/step - loss: 1.4998 - accuracy: 0.3359 - val_loss: 1.5368 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 13s 939ms/step - loss: 1.4756 - accuracy: 0.3321 - val_loss: 1.5473 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 15s 1s/step - loss: 1.4823 - accuracy: 0.2939 - val_loss: 1.5427 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 13s 860ms/step - loss: 1.4638 - accuracy: 0.3359 - val_loss: 1.5716 - val_accuracy: 0.3333
11/11 [==============================] - 2s 141ms/step
3/3 [==============================] - 0s 102ms/step
11/11 [==============================] - 1s 109ms/step
3/3 [==============================] - 1s 334ms/step
Model: "sequential_38"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_38 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_38 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_38 (LSTM)              (None, 200)               173600    
                                                                 
 dense_38 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 19s 933ms/step - loss: 1.5588 - accuracy: 0.2519 - val_loss: 1.5390 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 13s 894ms/step - loss: 1.5230 - accuracy: 0.3359 - val_loss: 1.5490 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 13s 882ms/step - loss: 1.4966 - accuracy: 0.3015 - val_loss: 1.5454 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 13s 909ms/step - loss: 1.4767 - accuracy: 0.3359 - val_loss: 1.5425 - val_accuracy: 0.3333
11/11 [==============================] - 2s 181ms/step
3/3 [==============================] - 1s 180ms/step
11/11 [==============================] - 1s 112ms/step
3/3 [==============================] - 0s 108ms/step
Model: "sequential_39"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_39 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_39 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_39 (LSTM)              (None, 200)               173600    
                                                                 
 dense_39 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 17s 995ms/step - loss: 1.5634 - accuracy: 0.2672 - val_loss: 1.5382 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 13s 953ms/step - loss: 1.5024 - accuracy: 0.3359 - val_loss: 1.5347 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 13s 948ms/step - loss: 1.4756 - accuracy: 0.3359 - val_loss: 1.5531 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 13s 949ms/step - loss: 1.4734 - accuracy: 0.3282 - val_loss: 1.5440 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 15s 1s/step - loss: 1.4627 - accuracy: 0.3359 - val_loss: 1.5701 - val_accuracy: 0.3333
11/11 [==============================] - 1s 109ms/step
3/3 [==============================] - 0s 101ms/step
11/11 [==============================] - 1s 114ms/step
3/3 [==============================] - 1s 173ms/step
Result of all runs:             model  train accuracy  test accuracy  train F1 score  \
0  Neural Network        0.335366       0.337349        0.168449   
1  Neural Network        0.335366       0.337349        0.168449   
2  Neural Network        0.335366       0.337349        0.168449   

   test F1 score  
0       0.170194  
1       0.170194  
2       0.170194  

LSTM_Model(X_train_wvfull_smote, X_test_wvfull, y_train_wvfull_smote, y_test_wvfull)

Model: "sequential_40"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_40 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_40 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_40 (LSTM)              (None, 200)               173600    
                                                                 
 dense_40 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 21s 856ms/step - loss: 1.5662 - accuracy: 0.2273 - val_loss: 1.9578 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 20s 913ms/step - loss: 1.5416 - accuracy: 0.2659 - val_loss: 2.2660 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 19s 849ms/step - loss: 1.5268 - accuracy: 0.2432 - val_loss: 2.5860 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 19s 874ms/step - loss: 1.5274 - accuracy: 0.2318 - val_loss: 2.4260 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 144ms/step
3/3 [==============================] - 0s 102ms/step
18/18 [==============================] - 2s 112ms/step
3/3 [==============================] - 0s 105ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	Neural Network	0.2	0.337349	0.066667	0.170194

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model(X_train_wvfull_smote, X_test_wvfull, y_train_wvfull_smote, y_test_wvfull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_41"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_41 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_41 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_41 (LSTM)              (None, 200)               173600    
                                                                 
 dense_41 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 23s 909ms/step - loss: 1.5696 - accuracy: 0.2295 - val_loss: 1.9466 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 18s 838ms/step - loss: 1.5425 - accuracy: 0.2750 - val_loss: 2.3704 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 22s 1s/step - loss: 1.5239 - accuracy: 0.2364 - val_loss: 2.5762 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 18s 845ms/step - loss: 1.5237 - accuracy: 0.2545 - val_loss: 2.4852 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 126ms/step
3/3 [==============================] - 1s 175ms/step
18/18 [==============================] - 3s 146ms/step
3/3 [==============================] - 0s 104ms/step
Model: "sequential_42"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_42 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_42 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_42 (LSTM)              (None, 200)               173600    
                                                                 
 dense_42 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 26s 1s/step - loss: 1.5649 - accuracy: 0.2341 - val_loss: 1.9206 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 21s 909ms/step - loss: 1.5465 - accuracy: 0.2682 - val_loss: 2.1606 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 20s 901ms/step - loss: 1.5295 - accuracy: 0.2227 - val_loss: 2.6596 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 18s 838ms/step - loss: 1.5292 - accuracy: 0.2386 - val_loss: 2.3681 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 111ms/step
3/3 [==============================] - 1s 169ms/step
18/18 [==============================] - 3s 155ms/step
3/3 [==============================] - 0s 104ms/step
Model: "sequential_43"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_43 (Embedding)    (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_43 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 lstm_43 (LSTM)              (None, 200)               173600    
                                                                 
 dense_43 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 177,805
Trainable params: 177,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
22/22 [==============================] - 22s 914ms/step - loss: 1.5625 - accuracy: 0.2227 - val_loss: 2.0211 - val_accuracy: 0.0000e+00
Epoch 2/100
22/22 [==============================] - 20s 922ms/step - loss: 1.5307 - accuracy: 0.2682 - val_loss: 2.0970 - val_accuracy: 0.0000e+00
Epoch 3/100
22/22 [==============================] - 18s 829ms/step - loss: 1.5336 - accuracy: 0.2386 - val_loss: 2.6710 - val_accuracy: 0.0000e+00
Epoch 4/100
22/22 [==============================] - 20s 917ms/step - loss: 1.5301 - accuracy: 0.2455 - val_loss: 2.3558 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 111ms/step
3/3 [==============================] - 0s 107ms/step
18/18 [==============================] - 2s 111ms/step
3/3 [==============================] - 0s 111ms/step
Result of all runs:             model  train accuracy  test accuracy  train F1 score  \
0  Neural Network             0.2       0.108434        0.066667   
1  Neural Network             0.2       0.108434        0.066667   
2  Neural Network             0.2       0.108434        0.066667   

   test F1 score  
0       0.021215  
1       0.021215  
2       0.021215  

Observations-

1. In general, LSTM method is giving accuracy around 33%. F1 score as 17%.
2. This is almost same for all 3 datasets i.e. countvectorizer, TFIDF, Word2vec.
3. If we compare with ANN model where each run gives different score, all runs in LSTM gives same accuracy and F1 score.
4. In general, smote dataset is not improving accuracy or F1 score.
5. We need to see more of tuning parameters to figure out best score possible in LSTM.

Tuning Method-

Let's apply tuning methods on LSTM model.

Each of dataset will be applied for tuning techniques. There are 2 tuning techniques used i.e. keras tuner and keras classifier. GridsearchCV and randomsearch methods are used.

There are 3 tuning parameters i.e. number of neurons, batch size and epoch number.

Each hidden layer is having relu activation function and output layer has Softmax activation function. Optimizer is Adam and loss function as categorical crossentropy. These are best options in a multiclass classification problem statement.

CountVectorizer Dataset-

Tuned_LSTM(X_cv_df, y_cv_df)

4/4 [==============================] - 3s 127ms/step - loss: 1.5961 - accuracy: 0.2575
1/1 [==============================] - 0s 460ms/step - loss: 1.5896 - accuracy: 0.2619
4/4 [==============================] - 2s 82ms/step - loss: 1.6084 - accuracy: 0.2216
1/1 [==============================] - 0s 308ms/step - loss: 1.6073 - accuracy: 0.3171
4/4 [==============================] - 3s 87ms/step - loss: 1.5984 - accuracy: 0.3297
1/1 [==============================] - 0s 331ms/step - loss: 1.5845 - accuracy: 0.3902
4/4 [==============================] - 2s 84ms/step - loss: 1.6065 - accuracy: 0.2378

WARNING:tensorflow:5 out of the last 28 calls to <function Model.make_test_function.<locals>.test_function at 0x7fe305e1fd00> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

1/1 [==============================] - 0s 322ms/step - loss: 1.6090 - accuracy: 0.1220
4/4 [==============================] - 2s 125ms/step - loss: 1.6109 - accuracy: 0.1297

WARNING:tensorflow:6 out of the last 29 calls to <function Model.make_test_function.<locals>.test_function at 0x7fe306501510> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

1/1 [==============================] - 0s 469ms/step - loss: 1.6093 - accuracy: 0.3171
4/4 [==============================] - 2s 83ms/step - loss: 1.6136 - accuracy: 0.1459
1/1 [==============================] - 0s 316ms/step - loss: 1.6118 - accuracy: 0.2683
4/4 [==============================] - 2s 91ms/step - loss: 1.6159 - accuracy: 0.2216
1/1 [==============================] - 0s 300ms/step - loss: 1.6081 - accuracy: 0.3171
4/4 [==============================] - 2s 83ms/step - loss: 1.6177 - accuracy: 0.1351
1/1 [==============================] - 0s 310ms/step - loss: 1.6114 - accuracy: 0.1463
4/4 [==============================] - 2s 83ms/step - loss: 1.6062 - accuracy: 0.3216
1/1 [==============================] - 0s 316ms/step - loss: 1.6098 - accuracy: 0.3171
4/4 [==============================] - 3s 134ms/step - loss: 1.6043 - accuracy: 0.2568
1/1 [==============================] - 0s 304ms/step - loss: 1.6018 - accuracy: 0.1463
4/4 [==============================] - 2s 88ms/step - loss: 1.6052 - accuracy: 0.2276
1/1 [==============================] - 0s 317ms/step - loss: 1.5988 - accuracy: 0.2381
4/4 [==============================] - 2s 88ms/step - loss: 1.6017 - accuracy: 0.2784
1/1 [==============================] - 0s 325ms/step - loss: 1.6012 - accuracy: 0.3415
4/4 [==============================] - 2s 90ms/step - loss: 1.6195 - accuracy: 0.2054
1/1 [==============================] - 0s 309ms/step - loss: 1.6184 - accuracy: 0.2195
4/4 [==============================] - 2s 154ms/step - loss: 1.6027 - accuracy: 0.3486
1/1 [==============================] - 1s 571ms/step - loss: 1.6109 - accuracy: 0.2195
4/4 [==============================] - 7s 287ms/step - loss: 1.6077 - accuracy: 0.3054
1/1 [==============================] - 1s 634ms/step - loss: 1.6032 - accuracy: 0.3415
4/4 [==============================] - 3s 157ms/step - loss: 1.6052 - accuracy: 0.2541
1/1 [==============================] - 0s 480ms/step - loss: 1.5986 - accuracy: 0.3659
4/4 [==============================] - 4s 181ms/step - loss: 1.6099 - accuracy: 0.2189
1/1 [==============================] - 1s 622ms/step - loss: 1.6017 - accuracy: 0.3171
4/4 [==============================] - 4s 146ms/step - loss: 1.6052 - accuracy: 0.2405
1/1 [==============================] - 1s 669ms/step - loss: 1.6052 - accuracy: 0.1463
4/4 [==============================] - 3s 141ms/step - loss: 1.6108 - accuracy: 0.2081
1/1 [==============================] - 0s 439ms/step - loss: 1.6101 - accuracy: 0.1220
4/4 [==============================] - 4s 224ms/step - loss: 1.6076 - accuracy: 0.2486
1/1 [==============================] - 1s 560ms/step - loss: 1.6045 - accuracy: 0.2195
4/4 [==============================] - 4s 251ms/step - loss: 1.6064 - accuracy: 0.2141
1/1 [==============================] - 0s 380ms/step - loss: 1.5979 - accuracy: 0.3095
4/4 [==============================] - 3s 183ms/step - loss: 1.6036 - accuracy: 0.2946
1/1 [==============================] - 0s 408ms/step - loss: 1.5983 - accuracy: 0.3415
4/4 [==============================] - 4s 249ms/step - loss: 1.6023 - accuracy: 0.3297
1/1 [==============================] - 1s 795ms/step - loss: 1.5924 - accuracy: 0.3902
4/4 [==============================] - 3s 134ms/step - loss: 1.6089 - accuracy: 0.2405
1/1 [==============================] - 0s 437ms/step - loss: 1.6088 - accuracy: 0.1220
4/4 [==============================] - 4s 160ms/step - loss: 1.6104 - accuracy: 0.1649
1/1 [==============================] - 1s 557ms/step - loss: 1.6075 - accuracy: 0.2439
4/4 [==============================] - 7s 206ms/step - loss: 1.6078 - accuracy: 0.2297
1/1 [==============================] - 0s 425ms/step - loss: 1.5990 - accuracy: 0.4390
4/4 [==============================] - 3s 200ms/step - loss: 1.6045 - accuracy: 0.3405
1/1 [==============================] - 1s 534ms/step - loss: 1.5990 - accuracy: 0.3171
4/4 [==============================] - 5s 233ms/step - loss: 1.6085 - accuracy: 0.2486
1/1 [==============================] - 1s 802ms/step - loss: 1.6024 - accuracy: 0.3415
4/4 [==============================] - 3s 126ms/step - loss: 1.6080 - accuracy: 0.2432
1/1 [==============================] - 1s 602ms/step - loss: 1.6115 - accuracy: 0.1220
4/4 [==============================] - 3s 142ms/step - loss: 1.6096 - accuracy: 0.2216
1/1 [==============================] - 0s 445ms/step - loss: 1.6039 - accuracy: 0.3659
4/4 [==============================] - 6s 476ms/step - loss: 1.6080 - accuracy: 0.2547
1/1 [==============================] - 1s 572ms/step - loss: 1.5994 - accuracy: 0.3571
4/4 [==============================] - 3s 284ms/step - loss: 1.6050 - accuracy: 0.2703
1/1 [==============================] - 1s 564ms/step - loss: 1.5956 - accuracy: 0.3415
4/4 [==============================] - 4s 408ms/step - loss: 1.6071 - accuracy: 0.2541
1/1 [==============================] - 1s 532ms/step - loss: 1.5993 - accuracy: 0.3902
4/4 [==============================] - 4s 269ms/step - loss: 1.6038 - accuracy: 0.2595
1/1 [==============================] - 0s 392ms/step - loss: 1.6086 - accuracy: 0.2195
4/4 [==============================] - 4s 379ms/step - loss: 1.6025 - accuracy: 0.3000
1/1 [==============================] - 1s 553ms/step - loss: 1.5902 - accuracy: 0.3415
4/4 [==============================] - 5s 366ms/step - loss: 1.6054 - accuracy: 0.3243
1/1 [==============================] - 1s 542ms/step - loss: 1.5952 - accuracy: 0.4390
4/4 [==============================] - 4s 277ms/step - loss: 1.6055 - accuracy: 0.3378
1/1 [==============================] - 1s 510ms/step - loss: 1.5971 - accuracy: 0.3171
4/4 [==============================] - 8s 453ms/step - loss: 1.6075 - accuracy: 0.2514
1/1 [==============================] - 1s 523ms/step - loss: 1.6004 - accuracy: 0.3415
4/4 [==============================] - 3s 204ms/step - loss: 1.6029 - accuracy: 0.2514
1/1 [==============================] - 0s 335ms/step - loss: 1.6111 - accuracy: 0.2927
4/4 [==============================] - 2s 205ms/step - loss: 1.6007 - accuracy: 0.2703
1/1 [==============================] - 0s 323ms/step - loss: 1.5924 - accuracy: 0.1463
4/4 [==============================] - 5s 769ms/step - loss: 1.6061 - accuracy: 0.2493
1/1 [==============================] - 0s 408ms/step - loss: 1.5958 - accuracy: 0.3571
4/4 [==============================] - 3s 426ms/step - loss: 1.6037 - accuracy: 0.3000
1/1 [==============================] - 0s 398ms/step - loss: 1.5960 - accuracy: 0.3415
4/4 [==============================] - 3s 425ms/step - loss: 1.6021 - accuracy: 0.2514
1/1 [==============================] - 1s 588ms/step - loss: 1.5838 - accuracy: 0.3902
4/4 [==============================] - 4s 420ms/step - loss: 1.6054 - accuracy: 0.3027
1/1 [==============================] - 0s 404ms/step - loss: 1.6103 - accuracy: 0.2195
4/4 [==============================] - 3s 422ms/step - loss: 1.6044 - accuracy: 0.3081
1/1 [==============================] - 0s 371ms/step - loss: 1.5917 - accuracy: 0.3415
4/4 [==============================] - 5s 837ms/step - loss: 1.6032 - accuracy: 0.2811
1/1 [==============================] - 1s 760ms/step - loss: 1.5881 - accuracy: 0.4390
4/4 [==============================] - 4s 415ms/step - loss: 1.6072 - accuracy: 0.1784
1/1 [==============================] - 0s 423ms/step - loss: 1.5971 - accuracy: 0.3415
4/4 [==============================] - 3s 426ms/step - loss: 1.6026 - accuracy: 0.3243
1/1 [==============================] - 0s 394ms/step - loss: 1.5963 - accuracy: 0.2927
4/4 [==============================] - 4s 637ms/step - loss: 1.6050 - accuracy: 0.2351
1/1 [==============================] - 1s 574ms/step - loss: 1.6092 - accuracy: 0.3415
4/4 [==============================] - 5s 430ms/step - loss: 1.6076 - accuracy: 0.2622
1/1 [==============================] - 0s 407ms/step - loss: 1.5985 - accuracy: 0.3659
4/4 [==============================] - 7s 1s/step - loss: 1.6059 - accuracy: 0.2547
1/1 [==============================] - 1s 801ms/step - loss: 1.5920 - accuracy: 0.3571
4/4 [==============================] - 6s 971ms/step - loss: 1.6004 - accuracy: 0.3108
1/1 [==============================] - 0s 485ms/step - loss: 1.5814 - accuracy: 0.3415
4/4 [==============================] - 9s 2s/step - loss: 1.6036 - accuracy: 0.2811
1/1 [==============================] - 1s 1s/step - loss: 1.5837 - accuracy: 0.3902
4/4 [==============================] - 10s 2s/step - loss: 1.5993 - accuracy: 0.3486
1/1 [==============================] - 1s 794ms/step - loss: 1.6055 - accuracy: 0.2195
4/4 [==============================] - 9s 2s/step - loss: 1.6025 - accuracy: 0.3351
1/1 [==============================] - 1s 625ms/step - loss: 1.5832 - accuracy: 0.3415
4/4 [==============================] - 5s 965ms/step - loss: 1.6043 - accuracy: 0.2243
1/1 [==============================] - 0s 464ms/step - loss: 1.5819 - accuracy: 0.4390
4/4 [==============================] - 11s 2s/step - loss: 1.6006 - accuracy: 0.3405
1/1 [==============================] - 1s 1s/step - loss: 1.5804 - accuracy: 0.3171
4/4 [==============================] - 12s 1s/step - loss: 1.6028 - accuracy: 0.2378
1/1 [==============================] - 1s 885ms/step - loss: 1.5970 - accuracy: 0.1463
4/4 [==============================] - 10s 1s/step - loss: 1.6028 - accuracy: 0.2865
1/1 [==============================] - 1s 785ms/step - loss: 1.6090 - accuracy: 0.2927
4/4 [==============================] - 10s 1s/step - loss: 1.6003 - accuracy: 0.3135
1/1 [==============================] - 1s 848ms/step - loss: 1.5819 - accuracy: 0.3659
4/4 [==============================] - 5s 216ms/step - loss: 1.6092 - accuracy: 0.2547
1/1 [==============================] - 1s 1s/step - loss: 1.6047 - accuracy: 0.2619
4/4 [==============================] - 7s 144ms/step - loss: 1.6107 - accuracy: 0.1405
1/1 [==============================] - 1s 656ms/step - loss: 1.6088 - accuracy: 0.3415
4/4 [==============================] - 5s 166ms/step - loss: 1.6094 - accuracy: 0.2514
1/1 [==============================] - 1s 608ms/step - loss: 1.6034 - accuracy: 0.2927
4/4 [==============================] - 4s 106ms/step - loss: 1.5955 - accuracy: 0.3486
1/1 [==============================] - 1s 548ms/step - loss: 1.6051 - accuracy: 0.2195
4/4 [==============================] - 4s 187ms/step - loss: 1.6038 - accuracy: 0.1324
1/1 [==============================] - 1s 1s/step - loss: 1.5994 - accuracy: 0.0244
4/4 [==============================] - 6s 153ms/step - loss: 1.5956 - accuracy: 0.3243
1/1 [==============================] - 1s 832ms/step - loss: 1.5862 - accuracy: 0.4390
4/4 [==============================] - 4s 156ms/step - loss: 1.6125 - accuracy: 0.0757
1/1 [==============================] - 1s 570ms/step - loss: 1.6093 - accuracy: 0.0244
4/4 [==============================] - 6s 154ms/step - loss: 1.5888 - accuracy: 0.3405
1/1 [==============================] - 0s 363ms/step - loss: 1.5925 - accuracy: 0.2927
4/4 [==============================] - 3s 149ms/step - loss: 1.6121 - accuracy: 0.2811
1/1 [==============================] - 1s 508ms/step - loss: 1.6052 - accuracy: 0.2927
4/4 [==============================] - 5s 183ms/step - loss: 1.6072 - accuracy: 0.2757
1/1 [==============================] - 1s 858ms/step - loss: 1.6059 - accuracy: 0.3659
4/4 [==============================] - 5s 205ms/step - loss: 1.6109 - accuracy: 0.1924
1/1 [==============================] - 1s 597ms/step - loss: 1.6045 - accuracy: 0.2381
4/4 [==============================] - 3s 197ms/step - loss: 1.6027 - accuracy: 0.2865
1/1 [==============================] - 1s 1s/step - loss: 1.5999 - accuracy: 0.3415
4/4 [==============================] - 7s 144ms/step - loss: 1.6094 - accuracy: 0.2216
1/1 [==============================] - 1s 738ms/step - loss: 1.6053 - accuracy: 0.4634
4/4 [==============================] - 7s 181ms/step - loss: 1.5991 - accuracy: 0.3486
1/1 [==============================] - 1s 851ms/step - loss: 1.6075 - accuracy: 0.2195
4/4 [==============================] - 4s 180ms/step - loss: 1.6098 - accuracy: 0.1757
1/1 [==============================] - 1s 978ms/step - loss: 1.6026 - accuracy: 0.2439
4/4 [==============================] - 6s 145ms/step - loss: 1.6038 - accuracy: 0.2730
1/1 [==============================] - 1s 752ms/step - loss: 1.5951 - accuracy: 0.4390
4/4 [==============================] - 5s 163ms/step - loss: 1.6078 - accuracy: 0.1324
1/1 [==============================] - 1s 653ms/step - loss: 1.6065 - accuracy: 0.3171
4/4 [==============================] - 5s 148ms/step - loss: 1.6030 - accuracy: 0.3405
1/1 [==============================] - 0s 423ms/step - loss: 1.5994 - accuracy: 0.2927
4/4 [==============================] - 3s 126ms/step - loss: 1.6031 - accuracy: 0.2676
1/1 [==============================] - 1s 511ms/step - loss: 1.6056 - accuracy: 0.1707
4/4 [==============================] - 3s 94ms/step - loss: 1.6087 - accuracy: 0.2703
1/1 [==============================] - 0s 394ms/step - loss: 1.6070 - accuracy: 0.1463
4/4 [==============================] - 4s 287ms/step - loss: 1.6091 - accuracy: 0.2114
1/1 [==============================] - 1s 1s/step - loss: 1.6019 - accuracy: 0.2857
4/4 [==============================] - 3s 152ms/step - loss: 1.6063 - accuracy: 0.2838
1/1 [==============================] - 0s 387ms/step - loss: 1.6047 - accuracy: 0.3415
4/4 [==============================] - 6s 203ms/step - loss: 1.6071 - accuracy: 0.3000
1/1 [==============================] - 1s 819ms/step - loss: 1.6001 - accuracy: 0.3902
4/4 [==============================] - 4s 132ms/step - loss: 1.6062 - accuracy: 0.3405
1/1 [==============================] - 0s 436ms/step - loss: 1.6084 - accuracy: 0.2195
4/4 [==============================] - 4s 224ms/step - loss: 1.6055 - accuracy: 0.2757
1/1 [==============================] - 1s 769ms/step - loss: 1.5999 - accuracy: 0.3415
4/4 [==============================] - 5s 219ms/step - loss: 1.6078 - accuracy: 0.2649
1/1 [==============================] - 1s 569ms/step - loss: 1.6040 - accuracy: 0.1951
4/4 [==============================] - 3s 176ms/step - loss: 1.6047 - accuracy: 0.2541
1/1 [==============================] - 0s 493ms/step - loss: 1.5964 - accuracy: 0.2683
4/4 [==============================] - 3s 181ms/step - loss: 1.6130 - accuracy: 0.1324
1/1 [==============================] - 0s 493ms/step - loss: 1.6060 - accuracy: 0.2927
4/4 [==============================] - 5s 189ms/step - loss: 1.6041 - accuracy: 0.3405
1/1 [==============================] - 1s 567ms/step - loss: 1.6068 - accuracy: 0.2927
4/4 [==============================] - 4s 194ms/step - loss: 1.6055 - accuracy: 0.2189
1/1 [==============================] - 1s 574ms/step - loss: 1.5969 - accuracy: 0.3415
4/4 [==============================] - 4s 483ms/step - loss: 1.6023 - accuracy: 0.3333
1/1 [==============================] - 1s 976ms/step - loss: 1.5916 - accuracy: 0.3571
4/4 [==============================] - 5s 432ms/step - loss: 1.6083 - accuracy: 0.2216
1/1 [==============================] - 1s 1s/step - loss: 1.5978 - accuracy: 0.4146
4/4 [==============================] - 11s 447ms/step - loss: 1.6094 - accuracy: 0.2054
1/1 [==============================] - 2s 2s/step - loss: 1.6011 - accuracy: 0.3902
4/4 [==============================] - 5s 522ms/step - loss: 1.5987 - accuracy: 0.3486
1/1 [==============================] - 1s 1s/step - loss: 1.6134 - accuracy: 0.2195
4/4 [==============================] - 6s 436ms/step - loss: 1.6066 - accuracy: 0.2351
1/1 [==============================] - 1s 1s/step - loss: 1.5948 - accuracy: 0.3415
4/4 [==============================] - 6s 524ms/step - loss: 1.6082 - accuracy: 0.1405
1/1 [==============================] - 0s 427ms/step - loss: 1.5999 - accuracy: 0.1951
4/4 [==============================] - 9s 603ms/step - loss: 1.6082 - accuracy: 0.2216
1/1 [==============================] - 2s 2s/step - loss: 1.5994 - accuracy: 0.3171
4/4 [==============================] - 5s 409ms/step - loss: 1.6081 - accuracy: 0.2297
1/1 [==============================] - 1s 732ms/step - loss: 1.6053 - accuracy: 0.1463
4/4 [==============================] - 7s 353ms/step - loss: 1.6049 - accuracy: 0.2784
1/1 [==============================] - 1s 839ms/step - loss: 1.6085 - accuracy: 0.2927
4/4 [==============================] - 5s 400ms/step - loss: 1.6031 - accuracy: 0.2730
1/1 [==============================] - 1s 642ms/step - loss: 1.5956 - accuracy: 0.3659
4/4 [==============================] - 7s 646ms/step - loss: 1.5993 - accuracy: 0.3333
1/1 [==============================] - 1s 1s/step - loss: 1.5865 - accuracy: 0.3571
4/4 [==============================] - 8s 906ms/step - loss: 1.6027 - accuracy: 0.3108
1/1 [==============================] - 1s 686ms/step - loss: 1.5945 - accuracy: 0.3415
4/4 [==============================] - 5s 564ms/step - loss: 1.6030 - accuracy: 0.3297
1/1 [==============================] - 1s 594ms/step - loss: 1.5887 - accuracy: 0.3902
4/4 [==============================] - 7s 811ms/step - loss: 1.5991 - accuracy: 0.3054
1/1 [==============================] - 1s 1s/step - loss: 1.6081 - accuracy: 0.2195
4/4 [==============================] - 8s 1s/step - loss: 1.6005 - accuracy: 0.3081
1/1 [==============================] - 1s 1s/step - loss: 1.5780 - accuracy: 0.3415
4/4 [==============================] - 5s 696ms/step - loss: 1.6054 - accuracy: 0.2216
1/1 [==============================] - 1s 731ms/step - loss: 1.5936 - accuracy: 0.4390
4/4 [==============================] - 7s 662ms/step - loss: 1.6020 - accuracy: 0.2189
1/1 [==============================] - 1s 583ms/step - loss: 1.5871 - accuracy: 0.2683
4/4 [==============================] - 7s 809ms/step - loss: 1.6031 - accuracy: 0.2135
1/1 [==============================] - 2s 2s/step - loss: 1.5981 - accuracy: 0.1463
4/4 [==============================] - 7s 774ms/step - loss: 1.6019 - accuracy: 0.3405
1/1 [==============================] - 1s 1s/step - loss: 1.6068 - accuracy: 0.2927
4/4 [==============================] - 11s 1s/step - loss: 1.6007 - accuracy: 0.2622
1/1 [==============================] - 0s 491ms/step - loss: 1.5899 - accuracy: 0.3659
4/4 [==============================] - 14s 2s/step - loss: 1.6061 - accuracy: 0.2547
1/1 [==============================] - 1s 855ms/step - loss: 1.5925 - accuracy: 0.3571
4/4 [==============================] - 10s 1s/step - loss: 1.5988 - accuracy: 0.2946
1/1 [==============================] - 1s 811ms/step - loss: 1.5797 - accuracy: 0.3415
4/4 [==============================] - 11s 1s/step - loss: 1.6007 - accuracy: 0.3081
1/1 [==============================] - 1s 1s/step - loss: 1.5769 - accuracy: 0.3902
4/4 [==============================] - 10s 2s/step - loss: 1.6012 - accuracy: 0.3243
1/1 [==============================] - 1s 812ms/step - loss: 1.6080 - accuracy: 0.2195
4/4 [==============================] - 10s 2s/step - loss: 1.6018 - accuracy: 0.3027
1/1 [==============================] - 1s 829ms/step - loss: 1.5785 - accuracy: 0.3415
4/4 [==============================] - 10s 2s/step - loss: 1.6021 - accuracy: 0.2270
1/1 [==============================] - 1s 834ms/step - loss: 1.5818 - accuracy: 0.2927
4/4 [==============================] - 10s 2s/step - loss: 1.6028 - accuracy: 0.2081
1/1 [==============================] - 1s 1s/step - loss: 1.5855 - accuracy: 0.3171
4/4 [==============================] - 12s 2s/step - loss: 1.6012 - accuracy: 0.2378
1/1 [==============================] - 1s 802ms/step - loss: 1.5945 - accuracy: 0.2927
4/4 [==============================] - 10s 2s/step - loss: 1.6003 - accuracy: 0.3432
1/1 [==============================] - 1s 775ms/step - loss: 1.6080 - accuracy: 0.2927
4/4 [==============================] - 12s 2s/step - loss: 1.6031 - accuracy: 0.2649
1/1 [==============================] - 2s 2s/step - loss: 1.5856 - accuracy: 0.3659
19/19 [==============================] - 10s 177ms/step - loss: 1.5963 - accuracy: 0.2575
3/3 [==============================] - 1s 29ms/step - loss: 1.5815 - accuracy: 0.2619
19/19 [==============================] - 6s 166ms/step - loss: 1.5858 - accuracy: 0.3351
3/3 [==============================] - 2s 34ms/step - loss: 1.5480 - accuracy: 0.3415
19/19 [==============================] - 9s 124ms/step - loss: 1.6044 - accuracy: 0.2946
3/3 [==============================] - 1s 37ms/step - loss: 1.5881 - accuracy: 0.3902
19/19 [==============================] - 8s 135ms/step - loss: 1.5953 - accuracy: 0.3486
3/3 [==============================] - 1s 38ms/step - loss: 1.6119 - accuracy: 0.2195
19/19 [==============================] - 8s 124ms/step - loss: 1.5912 - accuracy: 0.2703
3/3 [==============================] - 1s 32ms/step - loss: 1.5735 - accuracy: 0.2439
19/19 [==============================] - 5s 132ms/step - loss: 1.6067 - accuracy: 0.2838
3/3 [==============================] - 1s 40ms/step - loss: 1.6006 - accuracy: 0.4390
19/19 [==============================] - 8s 162ms/step - loss: 1.5962 - accuracy: 0.2216
3/3 [==============================] - 1s 32ms/step - loss: 1.5574 - accuracy: 0.3171
19/19 [==============================] - 9s 139ms/step - loss: 1.5952 - accuracy: 0.2811
3/3 [==============================] - 1s 35ms/step - loss: 1.5845 - accuracy: 0.2927
19/19 [==============================] - 5s 132ms/step - loss: 1.5842 - accuracy: 0.2351
3/3 [==============================] - 1s 42ms/step - loss: 1.6201 - accuracy: 0.1220
19/19 [==============================] - 7s 149ms/step - loss: 1.6035 - accuracy: 0.3270
3/3 [==============================] - 1s 34ms/step - loss: 1.5975 - accuracy: 0.3659
19/19 [==============================] - 6s 193ms/step - loss: 1.5942 - accuracy: 0.3008
3/3 [==============================] - 1s 35ms/step - loss: 1.5724 - accuracy: 0.3571
19/19 [==============================] - 11s 193ms/step - loss: 1.5961 - accuracy: 0.2459
3/3 [==============================] - 1s 33ms/step - loss: 1.5656 - accuracy: 0.3659
19/19 [==============================] - 6s 162ms/step - loss: 1.5973 - accuracy: 0.3108
3/3 [==============================] - 1s 32ms/step - loss: 1.5690 - accuracy: 0.3902
19/19 [==============================] - 6s 172ms/step - loss: 1.5938 - accuracy: 0.2486
3/3 [==============================] - 1s 23ms/step - loss: 1.6048 - accuracy: 0.2683
19/19 [==============================] - 6s 155ms/step - loss: 1.5977 - accuracy: 0.3108
3/3 [==============================] - 1s 26ms/step - loss: 1.5738 - accuracy: 0.2927
19/19 [==============================] - 9s 141ms/step - loss: 1.5998 - accuracy: 0.2595
3/3 [==============================] - 1s 15ms/step - loss: 1.5772 - accuracy: 0.1951
19/19 [==============================] - 7s 177ms/step - loss: 1.6005 - accuracy: 0.3351
3/3 [==============================] - 1s 27ms/step - loss: 1.5908 - accuracy: 0.3171
19/19 [==============================] - 5s 123ms/step - loss: 1.5922 - accuracy: 0.2405
3/3 [==============================] - 1s 32ms/step - loss: 1.5894 - accuracy: 0.1463
19/19 [==============================] - 5s 156ms/step - loss: 1.5944 - accuracy: 0.2568
3/3 [==============================] - 1s 32ms/step - loss: 1.6133 - accuracy: 0.2927
19/19 [==============================] - 5s 124ms/step - loss: 1.5906 - accuracy: 0.2189
3/3 [==============================] - 1s 31ms/step - loss: 1.5542 - accuracy: 0.3415
19/19 [==============================] - 7s 204ms/step - loss: 1.5919 - accuracy: 0.3306
3/3 [==============================] - 2s 56ms/step - loss: 1.5656 - accuracy: 0.3571
19/19 [==============================] - 7s 161ms/step - loss: 1.5906 - accuracy: 0.3135
3/3 [==============================] - 1s 42ms/step - loss: 1.5730 - accuracy: 0.3415
19/19 [==============================] - 9s 177ms/step - loss: 1.5917 - accuracy: 0.3297
3/3 [==============================] - 1s 18ms/step - loss: 1.5496 - accuracy: 0.3902
19/19 [==============================] - 8s 155ms/step - loss: 1.5871 - accuracy: 0.3486
3/3 [==============================] - 1s 31ms/step - loss: 1.6170 - accuracy: 0.2195
19/19 [==============================] - 8s 160ms/step - loss: 1.6001 - accuracy: 0.2973
3/3 [==============================] - 1s 53ms/step - loss: 1.5779 - accuracy: 0.3415
19/19 [==============================] - 5s 147ms/step - loss: 1.6040 - accuracy: 0.2162
3/3 [==============================] - 1s 29ms/step - loss: 1.5837 - accuracy: 0.4390
19/19 [==============================] - 7s 136ms/step - loss: 1.5901 - accuracy: 0.2243
3/3 [==============================] - 1s 24ms/step - loss: 1.5495 - accuracy: 0.3171
19/19 [==============================] - 6s 164ms/step - loss: 1.6003 - accuracy: 0.3108
3/3 [==============================] - 1s 24ms/step - loss: 1.5915 - accuracy: 0.2927
19/19 [==============================] - 5s 143ms/step - loss: 1.5915 - accuracy: 0.3216
3/3 [==============================] - 0s 17ms/step - loss: 1.6063 - accuracy: 0.2927
19/19 [==============================] - 3s 90ms/step - loss: 1.5911 - accuracy: 0.3027
3/3 [==============================] - 0s 22ms/step - loss: 1.5628 - accuracy: 0.3659
19/19 [==============================] - 4s 96ms/step - loss: 1.5957 - accuracy: 0.2683
3/3 [==============================] - 0s 20ms/step - loss: 1.5644 - accuracy: 0.3571
19/19 [==============================] - 3s 98ms/step - loss: 1.5932 - accuracy: 0.2892
3/3 [==============================] - 0s 21ms/step - loss: 1.5658 - accuracy: 0.3415
19/19 [==============================] - 5s 144ms/step - loss: 1.5731 - accuracy: 0.3243
3/3 [==============================] - 0s 19ms/step - loss: 1.4589 - accuracy: 0.3902
19/19 [==============================] - 3s 98ms/step - loss: 1.5857 - accuracy: 0.3243
3/3 [==============================] - 0s 18ms/step - loss: 1.6157 - accuracy: 0.2195
19/19 [==============================] - 5s 151ms/step - loss: 1.5903 - accuracy: 0.3243
3/3 [==============================] - 0s 19ms/step - loss: 1.5397 - accuracy: 0.3415
19/19 [==============================] - 3s 98ms/step - loss: 1.5836 - accuracy: 0.2568
3/3 [==============================] - 0s 17ms/step - loss: 1.5118 - accuracy: 0.4390
19/19 [==============================] - 4s 119ms/step - loss: 1.5982 - accuracy: 0.2054
3/3 [==============================] - 1s 25ms/step - loss: 1.5682 - accuracy: 0.3171
19/19 [==============================] - 4s 98ms/step - loss: 1.5872 - accuracy: 0.2405
3/3 [==============================] - 0s 20ms/step - loss: 1.5749 - accuracy: 0.1463
19/19 [==============================] - 3s 96ms/step - loss: 1.5985 - accuracy: 0.3027
3/3 [==============================] - 0s 19ms/step - loss: 1.6047 - accuracy: 0.2927
19/19 [==============================] - 4s 152ms/step - loss: 1.5968 - accuracy: 0.3162
3/3 [==============================] - 1s 29ms/step - loss: 1.5677 - accuracy: 0.3659
19/19 [==============================] - 5s 190ms/step - loss: 1.5769 - accuracy: 0.2818
3/3 [==============================] - 0s 41ms/step - loss: 1.4664 - accuracy: 0.3571
19/19 [==============================] - 6s 246ms/step - loss: 3077924192256.0000 - accuracy: 0.3189
3/3 [==============================] - 1s 69ms/step - loss: 3505665563289930694656.0000 - accuracy: 0.3415
19/19 [==============================] - 5s 196ms/step - loss: 1.5752 - accuracy: 0.3162
3/3 [==============================] - 0s 41ms/step - loss: 1.3883 - accuracy: 0.3902
19/19 [==============================] - 6s 246ms/step - loss: 1.5680 - accuracy: 0.3243
3/3 [==============================] - 1s 71ms/step - loss: 39478149120.0000 - accuracy: 0.2195
19/19 [==============================] - 7s 242ms/step - loss: 1.5838 - accuracy: 0.3216
3/3 [==============================] - 0s 41ms/step - loss: 1.4726 - accuracy: 0.3415
19/19 [==============================] - 7s 283ms/step - loss: 1.5817 - accuracy: 0.3054
3/3 [==============================] - 0s 41ms/step - loss: 1.3967 - accuracy: 0.4390
19/19 [==============================] - 6s 198ms/step - loss: 1.5847 - accuracy: 0.3054
3/3 [==============================] - 0s 41ms/step - loss: 1.4687 - accuracy: 0.3171
19/19 [==============================] - 7s 266ms/step - loss: 1.5731 - accuracy: 0.2622
3/3 [==============================] - 0s 41ms/step - loss: 9756784.0000 - accuracy: 0.2927
19/19 [==============================] - 5s 195ms/step - loss: 1.5758 - accuracy: 0.3189
3/3 [==============================] - 0s 43ms/step - loss: 2929235073171456.0000 - accuracy: 0.2927
19/19 [==============================] - 7s 260ms/step - loss: 1.5813 - accuracy: 0.3054
3/3 [==============================] - 0s 41ms/step - loss: 1.4689 - accuracy: 0.3659
19/19 [==============================] - 11s 523ms/step - loss: 1.5746 - accuracy: 0.2846
3/3 [==============================] - 1s 90ms/step - loss: 1.5326 - accuracy: 0.3571
19/19 [==============================] - 11s 489ms/step - loss: 1.5858 - accuracy: 0.3135
3/3 [==============================] - 1s 73ms/step - loss: 1.5582 - accuracy: 0.3415
19/19 [==============================] - 11s 479ms/step - loss: nan - accuracy: 0.2676
3/3 [==============================] - 1s 71ms/step - loss: nan - accuracy: 0.0488
19/19 [==============================] - 11s 485ms/step - loss: nan - accuracy: 0.2784
3/3 [==============================] - 1s 75ms/step - loss: nan - accuracy: 0.2683
19/19 [==============================] - 10s 471ms/step - loss: 1517366280192.0000 - accuracy: 0.3243
3/3 [==============================] - 1s 125ms/step - loss: 1.5232 - accuracy: 0.3415
19/19 [==============================] - 10s 440ms/step - loss: nan - accuracy: 0.2622
3/3 [==============================] - 1s 113ms/step - loss: nan - accuracy: 0.0732
19/19 [==============================] - 10s 440ms/step - loss: 1.5392 - accuracy: 0.3378
3/3 [==============================] - 1s 72ms/step - loss: 1.4876 - accuracy: 0.3171
19/19 [==============================] - 12s 504ms/step - loss: 1.5896 - accuracy: 0.2351
3/3 [==============================] - 1s 81ms/step - loss: 1.5795 - accuracy: 0.1463
19/19 [==============================] - 11s 481ms/step - loss: nan - accuracy: 0.2676
3/3 [==============================] - 1s 75ms/step - loss: nan - accuracy: 0.2439
19/19 [==============================] - 11s 472ms/step - loss: nan - accuracy: 0.2595
3/3 [==============================] - 1s 108ms/step - loss: nan - accuracy: 0.0732
19/19 [==============================] - 4s 84ms/step - loss: 1.6049 - accuracy: 0.2304
3/3 [==============================] - 0s 15ms/step - loss: 1.5968 - accuracy: 0.2381
19/19 [==============================] - 3s 73ms/step - loss: 1.6005 - accuracy: 0.2514
3/3 [==============================] - 0s 15ms/step - loss: 1.5867 - accuracy: 0.3659
19/19 [==============================] - 3s 100ms/step - loss: 1.6006 - accuracy: 0.2324
3/3 [==============================] - 1s 22ms/step - loss: 1.5858 - accuracy: 0.2195
19/19 [==============================] - 3s 72ms/step - loss: 1.5891 - accuracy: 0.3405
3/3 [==============================] - 0s 15ms/step - loss: 1.6029 - accuracy: 0.2195
19/19 [==============================] - 3s 73ms/step - loss: 1.5946 - accuracy: 0.2595
3/3 [==============================] - 0s 15ms/step - loss: 1.5749 - accuracy: 0.2439
19/19 [==============================] - 3s 76ms/step - loss: 1.6038 - accuracy: 0.2378
3/3 [==============================] - 1s 27ms/step - loss: 1.5890 - accuracy: 0.1951
19/19 [==============================] - 3s 73ms/step - loss: 1.5905 - accuracy: 0.2568
3/3 [==============================] - 0s 15ms/step - loss: 1.5738 - accuracy: 0.2683
19/19 [==============================] - 3s 72ms/step - loss: 1.6077 - accuracy: 0.2162
3/3 [==============================] - 0s 15ms/step - loss: 1.6038 - accuracy: 0.1463
19/19 [==============================] - 4s 111ms/step - loss: 1.5971 - accuracy: 0.3405
3/3 [==============================] - 0s 16ms/step - loss: 1.6028 - accuracy: 0.2927
19/19 [==============================] - 4s 72ms/step - loss: 1.5967 - accuracy: 0.3324
3/3 [==============================] - 0s 15ms/step - loss: 1.5894 - accuracy: 0.3659
19/19 [==============================] - 3s 77ms/step - loss: 1.6069 - accuracy: 0.2602
3/3 [==============================] - 0s 15ms/step - loss: 1.5987 - accuracy: 0.3571
19/19 [==============================] - 4s 114ms/step - loss: 1.6016 - accuracy: 0.2027
3/3 [==============================] - 0s 20ms/step - loss: 1.5911 - accuracy: 0.1707
19/19 [==============================] - 3s 77ms/step - loss: 1.5991 - accuracy: 0.3297
3/3 [==============================] - 0s 16ms/step - loss: 1.5784 - accuracy: 0.3902
19/19 [==============================] - 3s 75ms/step - loss: 1.5913 - accuracy: 0.3000
3/3 [==============================] - 1s 22ms/step - loss: 1.6078 - accuracy: 0.2195
19/19 [==============================] - 4s 77ms/step - loss: 1.5889 - accuracy: 0.3243
3/3 [==============================] - 0s 15ms/step - loss: 1.5538 - accuracy: 0.3415
19/19 [==============================] - 3s 77ms/step - loss: 1.6055 - accuracy: 0.2270
3/3 [==============================] - 0s 16ms/step - loss: 1.5904 - accuracy: 0.2683
19/19 [==============================] - 3s 76ms/step - loss: 1.5951 - accuracy: 0.3135
3/3 [==============================] - 0s 20ms/step - loss: 1.5713 - accuracy: 0.3171
19/19 [==============================] - 3s 76ms/step - loss: 1.6040 - accuracy: 0.2054
3/3 [==============================] - 0s 15ms/step - loss: 1.6017 - accuracy: 0.1707
19/19 [==============================] - 3s 77ms/step - loss: 1.5800 - accuracy: 0.2432
3/3 [==============================] - 0s 16ms/step - loss: 1.6296 - accuracy: 0.1220
19/19 [==============================] - 3s 76ms/step - loss: 1.5958 - accuracy: 0.2784
3/3 [==============================] - 0s 23ms/step - loss: 1.5692 - accuracy: 0.3659
19/19 [==============================] - 4s 94ms/step - loss: 1.5916 - accuracy: 0.3306
3/3 [==============================] - 0s 18ms/step - loss: 1.5706 - accuracy: 0.3571
19/19 [==============================] - 4s 85ms/step - loss: 1.5983 - accuracy: 0.2757
3/3 [==============================] - 0s 18ms/step - loss: 1.5781 - accuracy: 0.3415
19/19 [==============================] - 4s 126ms/step - loss: 1.6004 - accuracy: 0.2865
3/3 [==============================] - 1s 21ms/step - loss: 1.5823 - accuracy: 0.3902
19/19 [==============================] - 3s 85ms/step - loss: 1.5967 - accuracy: 0.3081
3/3 [==============================] - 0s 17ms/step - loss: 1.6031 - accuracy: 0.2195
19/19 [==============================] - 3s 85ms/step - loss: 1.5959 - accuracy: 0.3027
3/3 [==============================] - 0s 16ms/step - loss: 1.5614 - accuracy: 0.3415
19/19 [==============================] - 4s 110ms/step - loss: 1.5979 - accuracy: 0.3108
3/3 [==============================] - 1s 25ms/step - loss: 1.5740 - accuracy: 0.4390
19/19 [==============================] - 3s 85ms/step - loss: 1.5925 - accuracy: 0.2216
3/3 [==============================] - 0s 20ms/step - loss: 1.5489 - accuracy: 0.3171
19/19 [==============================] - 3s 83ms/step - loss: 1.5908 - accuracy: 0.2162
3/3 [==============================] - 0s 18ms/step - loss: 1.5800 - accuracy: 0.2927
19/19 [==============================] - 4s 123ms/step - loss: 1.5935 - accuracy: 0.2973
3/3 [==============================] - 1s 22ms/step - loss: 1.6062 - accuracy: 0.2927
19/19 [==============================] - 3s 85ms/step - loss: 1.5841 - accuracy: 0.3324
3/3 [==============================] - 0s 23ms/step - loss: 1.5589 - accuracy: 0.3659
19/19 [==============================] - 3s 100ms/step - loss: 1.5732 - accuracy: 0.2873
3/3 [==============================] - 0s 20ms/step - loss: 1.4880 - accuracy: 0.3571
19/19 [==============================] - 4s 143ms/step - loss: 1.5823 - accuracy: 0.3108
3/3 [==============================] - 0s 31ms/step - loss: 1.5336 - accuracy: 0.3415
19/19 [==============================] - 3s 99ms/step - loss: 1.5787 - accuracy: 0.3297
3/3 [==============================] - 1s 21ms/step - loss: 1.4839 - accuracy: 0.3902
19/19 [==============================] - 4s 131ms/step - loss: 1.5849 - accuracy: 0.3378
3/3 [==============================] - 1s 31ms/step - loss: 1.6119 - accuracy: 0.2195
19/19 [==============================] - 4s 100ms/step - loss: 1.5909 - accuracy: 0.3000
3/3 [==============================] - 0s 19ms/step - loss: 1.5334 - accuracy: 0.3415
19/19 [==============================] - 5s 143ms/step - loss: 1.6011 - accuracy: 0.2568
3/3 [==============================] - 0s 21ms/step - loss: 1.5776 - accuracy: 0.4390
19/19 [==============================] - 5s 131ms/step - loss: 1.5868 - accuracy: 0.2541
3/3 [==============================] - 0s 19ms/step - loss: 1.5272 - accuracy: 0.3171
19/19 [==============================] - 3s 100ms/step - loss: 1.5934 - accuracy: 0.2108
3/3 [==============================] - 0s 20ms/step - loss: 1.5812 - accuracy: 0.1463
19/19 [==============================] - 3s 98ms/step - loss: 1.5871 - accuracy: 0.3270
3/3 [==============================] - 1s 28ms/step - loss: 1.6083 - accuracy: 0.2927
19/19 [==============================] - 4s 108ms/step - loss: 1.5906 - accuracy: 0.3324
3/3 [==============================] - 0s 20ms/step - loss: 1.5622 - accuracy: 0.3659
19/19 [==============================] - 5s 197ms/step - loss: 1.5839 - accuracy: 0.2629
3/3 [==============================] - 1s 76ms/step - loss: 1.4814 - accuracy: 0.3571
19/19 [==============================] - 6s 200ms/step - loss: 105064352.0000 - accuracy: 0.3081
3/3 [==============================] - 0s 43ms/step - loss: 1.4342 - accuracy: 0.3415
19/19 [==============================] - 5s 201ms/step - loss: 1.5770 - accuracy: 0.3081
3/3 [==============================] - 1s 62ms/step - loss: 1.3904 - accuracy: 0.3902
19/19 [==============================] - 6s 198ms/step - loss: 1522872670176149504.0000 - accuracy: 0.3378
3/3 [==============================] - 0s 42ms/step - loss: nan - accuracy: 0.2683
19/19 [==============================] - 8s 289ms/step - loss: 1.5569 - accuracy: 0.3162
3/3 [==============================] - 0s 43ms/step - loss: 1.4744 - accuracy: 0.3415
19/19 [==============================] - 7s 290ms/step - loss: 1.5838 - accuracy: 0.3108
3/3 [==============================] - 0s 41ms/step - loss: 1.4660 - accuracy: 0.4390
19/19 [==============================] - 5s 205ms/step - loss: 1.6622 - accuracy: 0.3324
3/3 [==============================] - 0s 40ms/step - loss: 1.4752 - accuracy: 0.3171
19/19 [==============================] - 7s 264ms/step - loss: 1.5764 - accuracy: 0.2622
3/3 [==============================] - 0s 43ms/step - loss: 1.5617 - accuracy: 0.2927
19/19 [==============================] - 5s 205ms/step - loss: 289843806208.0000 - accuracy: 0.3270
3/3 [==============================] - 0s 41ms/step - loss: 1.7227 - accuracy: 0.2927
19/19 [==============================] - 7s 250ms/step - loss: 21306020642947072.0000 - accuracy: 0.3324
3/3 [==============================] - 0s 41ms/step - loss: 1.4802 - accuracy: 0.3659
19/19 [==============================] - 12s 528ms/step - loss: 12254939.0000 - accuracy: 0.2764
3/3 [==============================] - 1s 83ms/step - loss: 1.5550 - accuracy: 0.3571
19/19 [==============================] - 12s 529ms/step - loss: nan - accuracy: 0.2541
3/3 [==============================] - 1s 75ms/step - loss: nan - accuracy: 0.0244
19/19 [==============================] - 11s 495ms/step - loss: 7409258819747840.0000 - accuracy: 0.3081
3/3 [==============================] - 1s 114ms/step - loss: 1.5011 - accuracy: 0.3902
19/19 [==============================] - 10s 459ms/step - loss: nan - accuracy: 0.2459
3/3 [==============================] - 1s 121ms/step - loss: nan - accuracy: 0.2683
19/19 [==============================] - 11s 449ms/step - loss: 155.9097 - accuracy: 0.2919
3/3 [==============================] - 1s 132ms/step - loss: 1.5398 - accuracy: 0.3415
19/19 [==============================] - 11s 455ms/step - loss: nan - accuracy: 0.2135
3/3 [==============================] - 1s 79ms/step - loss: nan - accuracy: 0.0732
19/19 [==============================] - 12s 448ms/step - loss: nan - accuracy: 0.2486
3/3 [==============================] - 1s 127ms/step - loss: nan - accuracy: 0.0732
19/19 [==============================] - 10s 447ms/step - loss: 278.1500 - accuracy: 0.2351
3/3 [==============================] - 1s 108ms/step - loss: 1.5755 - accuracy: 0.2927
19/19 [==============================] - 10s 459ms/step - loss: 26720419840.0000 - accuracy: 0.3378
3/3 [==============================] - 1s 122ms/step - loss: 1.6120 - accuracy: 0.2927
19/19 [==============================] - 11s 450ms/step - loss: 1.5646 - accuracy: 0.3162
3/3 [==============================] - 1s 123ms/step - loss: 1.5659 - accuracy: 0.3659
8/8 [==============================] - 2s 80ms/step - loss: 1.6106 - accuracy: 0.1653
1/1 [==============================] - 0s 306ms/step - loss: 1.6046 - accuracy: 0.2619
8/8 [==============================] - 2s 82ms/step - loss: 1.6035 - accuracy: 0.2378
1/1 [==============================] - 0s 319ms/step - loss: 1.5971 - accuracy: 0.2195
8/8 [==============================] - 2s 81ms/step - loss: 1.6083 - accuracy: 0.2054
1/1 [==============================] - 0s 316ms/step - loss: 1.6033 - accuracy: 0.1951
8/8 [==============================] - 3s 116ms/step - loss: 1.6065 - accuracy: 0.2541
1/1 [==============================] - 0s 450ms/step - loss: 1.6060 - accuracy: 0.2683
8/8 [==============================] - 2s 81ms/step - loss: 1.6098 - accuracy: 0.2162
1/1 [==============================] - 0s 302ms/step - loss: 1.6042 - accuracy: 0.3659
8/8 [==============================] - 2s 83ms/step - loss: 1.6093 - accuracy: 0.2514
1/1 [==============================] - 0s 309ms/step - loss: 1.6040 - accuracy: 0.1951
8/8 [==============================] - 2s 78ms/step - loss: 1.6039 - accuracy: 0.2216
1/1 [==============================] - 0s 314ms/step - loss: 1.5916 - accuracy: 0.3171
8/8 [==============================] - 2s 113ms/step - loss: 1.6065 - accuracy: 0.3459
1/1 [==============================] - 0s 431ms/step - loss: 1.6046 - accuracy: 0.2927
8/8 [==============================] - 4s 82ms/step - loss: 1.6002 - accuracy: 0.3432
1/1 [==============================] - 0s 318ms/step - loss: 1.6066 - accuracy: 0.2927
8/8 [==============================] - 2s 80ms/step - loss: 1.6043 - accuracy: 0.3000
1/1 [==============================] - 0s 328ms/step - loss: 1.5950 - accuracy: 0.3659
8/8 [==============================] - 2s 86ms/step - loss: 1.6096 - accuracy: 0.2087
1/1 [==============================] - 0s 340ms/step - loss: 1.6044 - accuracy: 0.3571
8/8 [==============================] - 3s 121ms/step - loss: 1.5993 - accuracy: 0.2378
1/1 [==============================] - 0s 500ms/step - loss: 1.5842 - accuracy: 0.1707
8/8 [==============================] - 2s 84ms/step - loss: 1.6075 - accuracy: 0.2514
1/1 [==============================] - 0s 307ms/step - loss: 1.5912 - accuracy: 0.3171
8/8 [==============================] - 2s 90ms/step - loss: 1.6069 - accuracy: 0.3216
1/1 [==============================] - 0s 322ms/step - loss: 1.6052 - accuracy: 0.2195
8/8 [==============================] - 2s 86ms/step - loss: 1.6108 - accuracy: 0.1351
1/1 [==============================] - 0s 305ms/step - loss: 1.6047 - accuracy: 0.3659
8/8 [==============================] - 3s 130ms/step - loss: 1.6101 - accuracy: 0.1378
1/1 [==============================] - 0s 359ms/step - loss: 1.6075 - accuracy: 0.2683
8/8 [==============================] - 2s 86ms/step - loss: 1.6020 - accuracy: 0.3108
1/1 [==============================] - 0s 305ms/step - loss: 1.5889 - accuracy: 0.3171
8/8 [==============================] - 2s 87ms/step - loss: 1.6060 - accuracy: 0.3162
1/1 [==============================] - 0s 310ms/step - loss: 1.6008 - accuracy: 0.2927
8/8 [==============================] - 2s 86ms/step - loss: 1.6086 - accuracy: 0.2216
1/1 [==============================] - 0s 450ms/step - loss: 1.6113 - accuracy: 0.1220
8/8 [==============================] - 3s 84ms/step - loss: 1.6113 - accuracy: 0.2270
1/1 [==============================] - 0s 301ms/step - loss: 1.5919 - accuracy: 0.3659
8/8 [==============================] - 2s 94ms/step - loss: 1.6077 - accuracy: 0.2575
1/1 [==============================] - 0s 337ms/step - loss: 1.6001 - accuracy: 0.2619
8/8 [==============================] - 2s 95ms/step - loss: 1.6013 - accuracy: 0.2378
1/1 [==============================] - 0s 321ms/step - loss: 1.5923 - accuracy: 0.1707
8/8 [==============================] - 3s 144ms/step - loss: 1.6041 - accuracy: 0.2676
1/1 [==============================] - 0s 499ms/step - loss: 1.5911 - accuracy: 0.4390
8/8 [==============================] - 2s 96ms/step - loss: 1.5956 - accuracy: 0.3108
1/1 [==============================] - 0s 316ms/step - loss: 1.6086 - accuracy: 0.2195
8/8 [==============================] - 2s 97ms/step - loss: 1.6017 - accuracy: 0.3351
1/1 [==============================] - 0s 322ms/step - loss: 1.5897 - accuracy: 0.3415
8/8 [==============================] - 2s 97ms/step - loss: 1.6056 - accuracy: 0.1919
1/1 [==============================] - 0s 316ms/step - loss: 1.5919 - accuracy: 0.2683
8/8 [==============================] - 3s 140ms/step - loss: 1.5987 - accuracy: 0.3378
1/1 [==============================] - 0s 500ms/step - loss: 1.5849 - accuracy: 0.3171
8/8 [==============================] - 2s 94ms/step - loss: 1.5998 - accuracy: 0.1973
1/1 [==============================] - 0s 317ms/step - loss: 1.5958 - accuracy: 0.2927
8/8 [==============================] - 2s 94ms/step - loss: 1.6056 - accuracy: 0.2351
1/1 [==============================] - 0s 335ms/step - loss: 1.6077 - accuracy: 0.2439
8/8 [==============================] - 3s 129ms/step - loss: 1.6049 - accuracy: 0.2919
1/1 [==============================] - 0s 458ms/step - loss: 1.5989 - accuracy: 0.3659
8/8 [==============================] - 3s 126ms/step - loss: 1.6033 - accuracy: 0.2331
1/1 [==============================] - 0s 338ms/step - loss: 1.5878 - accuracy: 0.2619
8/8 [==============================] - 4s 126ms/step - loss: 1.5963 - accuracy: 0.3351
1/1 [==============================] - 0s 334ms/step - loss: 1.5782 - accuracy: 0.3415
8/8 [==============================] - 3s 127ms/step - loss: 1.6009 - accuracy: 0.2838
1/1 [==============================] - 0s 353ms/step - loss: 1.5789 - accuracy: 0.3902
8/8 [==============================] - 4s 176ms/step - loss: 1.5984 - accuracy: 0.2351
1/1 [==============================] - 0s 330ms/step - loss: 1.6132 - accuracy: 0.2195
8/8 [==============================] - 3s 125ms/step - loss: 1.6023 - accuracy: 0.2973
1/1 [==============================] - 0s 321ms/step - loss: 1.5813 - accuracy: 0.3415
8/8 [==============================] - 3s 125ms/step - loss: 1.6032 - accuracy: 0.2189
1/1 [==============================] - 0s 325ms/step - loss: 1.5808 - accuracy: 0.4390
8/8 [==============================] - 3s 172ms/step - loss: 1.6029 - accuracy: 0.1892
1/1 [==============================] - 0s 458ms/step - loss: 1.5831 - accuracy: 0.3171
8/8 [==============================] - 3s 120ms/step - loss: 1.6045 - accuracy: 0.1514
1/1 [==============================] - 0s 326ms/step - loss: 1.6003 - accuracy: 0.2927
8/8 [==============================] - 3s 126ms/step - loss: 1.5983 - accuracy: 0.3432
1/1 [==============================] - 0s 319ms/step - loss: 1.6065 - accuracy: 0.2927
8/8 [==============================] - 3s 125ms/step - loss: 1.5968 - accuracy: 0.3324
1/1 [==============================] - 0s 334ms/step - loss: 1.5813 - accuracy: 0.3659
8/8 [==============================] - 5s 434ms/step - loss: 1.5996 - accuracy: 0.2954
1/1 [==============================] - 0s 393ms/step - loss: 1.5779 - accuracy: 0.3571
8/8 [==============================] - 4s 309ms/step - loss: 1.6022 - accuracy: 0.3135
1/1 [==============================] - 0s 386ms/step - loss: 1.5882 - accuracy: 0.3415
8/8 [==============================] - 6s 547ms/step - loss: 1.5987 - accuracy: 0.3189
1/1 [==============================] - 0s 393ms/step - loss: 1.5714 - accuracy: 0.3902
8/8 [==============================] - 5s 310ms/step - loss: 1.5964 - accuracy: 0.3486
1/1 [==============================] - 0s 430ms/step - loss: 1.6118 - accuracy: 0.2195
8/8 [==============================] - 6s 544ms/step - loss: 1.5984 - accuracy: 0.2838
1/1 [==============================] - 0s 411ms/step - loss: 1.5663 - accuracy: 0.3415
8/8 [==============================] - 4s 321ms/step - loss: 1.5984 - accuracy: 0.3162
1/1 [==============================] - 0s 383ms/step - loss: 1.5714 - accuracy: 0.4390
8/8 [==============================] - 6s 522ms/step - loss: 1.6004 - accuracy: 0.3027
1/1 [==============================] - 0s 370ms/step - loss: 1.5769 - accuracy: 0.3171
8/8 [==============================] - 4s 313ms/step - loss: 1.6016 - accuracy: 0.2216
1/1 [==============================] - 0s 392ms/step - loss: 1.5925 - accuracy: 0.1951
8/8 [==============================] - 4s 317ms/step - loss: 1.6011 - accuracy: 0.2919
1/1 [==============================] - 1s 619ms/step - loss: 1.6058 - accuracy: 0.2927
8/8 [==============================] - 5s 309ms/step - loss: 1.5998 - accuracy: 0.3000
1/1 [==============================] - 0s 408ms/step - loss: 1.5765 - accuracy: 0.3659
8/8 [==============================] - 9s 944ms/step - loss: 1.5905 - accuracy: 0.2656
1/1 [==============================] - 0s 491ms/step - loss: 1.5171 - accuracy: 0.3571
8/8 [==============================] - 7s 674ms/step - loss: 1.5903 - accuracy: 0.3351
1/1 [==============================] - 0s 492ms/step - loss: 1.5320 - accuracy: 0.3415
8/8 [==============================] - 9s 712ms/step - loss: 1.5929 - accuracy: 0.3054
1/1 [==============================] - 1s 510ms/step - loss: 1.5335 - accuracy: 0.3902
8/8 [==============================] - 8s 754ms/step - loss: 1.5900 - accuracy: 0.3216
1/1 [==============================] - 0s 462ms/step - loss: 1.6139 - accuracy: 0.2195
8/8 [==============================] - 9s 952ms/step - loss: 1.5904 - accuracy: 0.3216
1/1 [==============================] - 1s 509ms/step - loss: 1.5246 - accuracy: 0.3415
8/8 [==============================] - 10s 907ms/step - loss: 1.5936 - accuracy: 0.2892
1/1 [==============================] - 1s 577ms/step - loss: 1.5232 - accuracy: 0.4390
8/8 [==============================] - 8s 700ms/step - loss: 1.5950 - accuracy: 0.3243
1/1 [==============================] - 1s 623ms/step - loss: 1.5429 - accuracy: 0.3171
8/8 [==============================] - 9s 751ms/step - loss: 1.5920 - accuracy: 0.2405
1/1 [==============================] - 1s 502ms/step - loss: 1.5684 - accuracy: 0.1463
8/8 [==============================] - 9s 956ms/step - loss: 1.5993 - accuracy: 0.2081
1/1 [==============================] - 1s 523ms/step - loss: 1.6167 - accuracy: 0.1220
8/8 [==============================] - 7s 689ms/step - loss: 1.5902 - accuracy: 0.3324
1/1 [==============================] - 1s 761ms/step - loss: 1.5285 - accuracy: 0.3659
8/8 [==============================] - 3s 82ms/step - loss: 1.6061 - accuracy: 0.3225
1/1 [==============================] - 0s 302ms/step - loss: 1.5993 - accuracy: 0.3571
8/8 [==============================] - 2s 85ms/step - loss: 1.6015 - accuracy: 0.2378
1/1 [==============================] - 0s 332ms/step - loss: 1.5959 - accuracy: 0.1707
8/8 [==============================] - 2s 79ms/step - loss: 1.6117 - accuracy: 0.2324
1/1 [==============================] - 0s 327ms/step - loss: 1.6062 - accuracy: 0.2195
8/8 [==============================] - 2s 81ms/step - loss: 1.5972 - accuracy: 0.2568
1/1 [==============================] - 0s 419ms/step - loss: 1.6079 - accuracy: 0.2683
8/8 [==============================] - 3s 79ms/step - loss: 1.6090 - accuracy: 0.2081
1/1 [==============================] - 0s 306ms/step - loss: 1.6045 - accuracy: 0.3415
8/8 [==============================] - 2s 79ms/step - loss: 1.6062 - accuracy: 0.3081
1/1 [==============================] - 0s 305ms/step - loss: 1.5888 - accuracy: 0.4390
8/8 [==============================] - 3s 83ms/step - loss: 1.6074 - accuracy: 0.2108
1/1 [==============================] - 0s 328ms/step - loss: 1.5982 - accuracy: 0.2927
8/8 [==============================] - 3s 124ms/step - loss: 1.5965 - accuracy: 0.3405
1/1 [==============================] - 0s 473ms/step - loss: 1.5970 - accuracy: 0.2927
8/8 [==============================] - 2s 82ms/step - loss: 1.6060 - accuracy: 0.2676
1/1 [==============================] - 0s 316ms/step - loss: 1.6125 - accuracy: 0.1707
8/8 [==============================] - 2s 81ms/step - loss: 1.6061 - accuracy: 0.2703
1/1 [==============================] - 0s 318ms/step - loss: 1.6083 - accuracy: 0.1463
8/8 [==============================] - 2s 94ms/step - loss: 1.6001 - accuracy: 0.3333
1/1 [==============================] - 0s 334ms/step - loss: 1.5910 - accuracy: 0.3571
8/8 [==============================] - 3s 135ms/step - loss: 1.6065 - accuracy: 0.2459
1/1 [==============================] - 0s 487ms/step - loss: 1.5942 - accuracy: 0.3659
8/8 [==============================] - 3s 103ms/step - loss: 1.6051 - accuracy: 0.3135
1/1 [==============================] - 0s 371ms/step - loss: 1.5965 - accuracy: 0.3902
8/8 [==============================] - 3s 94ms/step - loss: 1.6027 - accuracy: 0.3162
1/1 [==============================] - 0s 318ms/step - loss: 1.6059 - accuracy: 0.2195
8/8 [==============================] - 2s 116ms/step - loss: 1.6093 - accuracy: 0.2432
1/1 [==============================] - 1s 549ms/step - loss: 1.6036 - accuracy: 0.3415
8/8 [==============================] - 3s 86ms/step - loss: 1.6060 - accuracy: 0.2676
1/1 [==============================] - 0s 357ms/step - loss: 1.5959 - accuracy: 0.4390
8/8 [==============================] - 2s 90ms/step - loss: 1.5985 - accuracy: 0.2405
1/1 [==============================] - 0s 309ms/step - loss: 1.5821 - accuracy: 0.3415
8/8 [==============================] - 2s 87ms/step - loss: 1.6065 - accuracy: 0.2378
1/1 [==============================] - 0s 336ms/step - loss: 1.6059 - accuracy: 0.2195
8/8 [==============================] - 4s 113ms/step - loss: 1.6047 - accuracy: 0.2432
1/1 [==============================] - 0s 320ms/step - loss: 1.6125 - accuracy: 0.1220
8/8 [==============================] - 2s 88ms/step - loss: 1.5959 - accuracy: 0.2703
1/1 [==============================] - 0s 328ms/step - loss: 1.5960 - accuracy: 0.1463
8/8 [==============================] - 2s 99ms/step - loss: 1.6055 - accuracy: 0.2764
1/1 [==============================] - 0s 322ms/step - loss: 1.5973 - accuracy: 0.3571
8/8 [==============================] - 2s 93ms/step - loss: 1.6015 - accuracy: 0.2919
1/1 [==============================] - 0s 480ms/step - loss: 1.5898 - accuracy: 0.3415
8/8 [==============================] - 3s 94ms/step - loss: 1.6077 - accuracy: 0.2405
1/1 [==============================] - 0s 310ms/step - loss: 1.5998 - accuracy: 0.3902
8/8 [==============================] - 3s 109ms/step - loss: 1.6044 - accuracy: 0.2595
1/1 [==============================] - 0s 329ms/step - loss: 1.6052 - accuracy: 0.2195
8/8 [==============================] - 2s 107ms/step - loss: 1.6066 - accuracy: 0.2432
1/1 [==============================] - 0s 370ms/step - loss: 1.5944 - accuracy: 0.3415
8/8 [==============================] - 3s 148ms/step - loss: 1.6000 - accuracy: 0.2649
1/1 [==============================] - 0s 467ms/step - loss: 1.5834 - accuracy: 0.1951
8/8 [==============================] - 3s 94ms/step - loss: 1.6068 - accuracy: 0.2189
1/1 [==============================] - 0s 322ms/step - loss: 1.5975 - accuracy: 0.3171
8/8 [==============================] - 2s 95ms/step - loss: 1.6056 - accuracy: 0.3216
1/1 [==============================] - 0s 325ms/step - loss: 1.5989 - accuracy: 0.2927
8/8 [==============================] - 2s 95ms/step - loss: 1.5958 - accuracy: 0.3405
1/1 [==============================] - 0s 315ms/step - loss: 1.6066 - accuracy: 0.2927
8/8 [==============================] - 3s 143ms/step - loss: 1.5994 - accuracy: 0.2703
1/1 [==============================] - 1s 502ms/step - loss: 1.5980 - accuracy: 0.1463
8/8 [==============================] - 3s 124ms/step - loss: 1.6065 - accuracy: 0.2249
1/1 [==============================] - 0s 322ms/step - loss: 1.5929 - accuracy: 0.3571
8/8 [==============================] - 3s 125ms/step - loss: 1.6014 - accuracy: 0.3027
1/1 [==============================] - 0s 333ms/step - loss: 1.5859 - accuracy: 0.3415
8/8 [==============================] - 3s 123ms/step - loss: 1.6058 - accuracy: 0.2838
1/1 [==============================] - 0s 460ms/step - loss: 1.5952 - accuracy: 0.3902
8/8 [==============================] - 3s 127ms/step - loss: 1.6023 - accuracy: 0.3135
1/1 [==============================] - 0s 336ms/step - loss: 1.6064 - accuracy: 0.2195
8/8 [==============================] - 3s 127ms/step - loss: 1.6012 - accuracy: 0.3135
1/1 [==============================] - 0s 359ms/step - loss: 1.5790 - accuracy: 0.3415
8/8 [==============================] - 3s 196ms/step - loss: 1.6033 - accuracy: 0.2919
1/1 [==============================] - 1s 596ms/step - loss: 1.5824 - accuracy: 0.4390
8/8 [==============================] - 4s 158ms/step - loss: 1.5985 - accuracy: 0.2919
1/1 [==============================] - 0s 356ms/step - loss: 1.5791 - accuracy: 0.3171
8/8 [==============================] - 3s 134ms/step - loss: 1.6044 - accuracy: 0.2649
1/1 [==============================] - 0s 359ms/step - loss: 1.5962 - accuracy: 0.2927
8/8 [==============================] - 3s 126ms/step - loss: 1.6026 - accuracy: 0.3243
1/1 [==============================] - 0s 334ms/step - loss: 1.6059 - accuracy: 0.2927
8/8 [==============================] - 3s 188ms/step - loss: 1.5942 - accuracy: 0.3324
1/1 [==============================] - 1s 516ms/step - loss: 1.5770 - accuracy: 0.3659
8/8 [==============================] - 4s 296ms/step - loss: 1.5986 - accuracy: 0.2629
1/1 [==============================] - 0s 391ms/step - loss: 1.5761 - accuracy: 0.3571
8/8 [==============================] - 5s 319ms/step - loss: 1.5948 - accuracy: 0.2946
1/1 [==============================] - 0s 380ms/step - loss: 1.5696 - accuracy: 0.3415
8/8 [==============================] - 4s 314ms/step - loss: 1.5958 - accuracy: 0.2919
1/1 [==============================] - 0s 421ms/step - loss: 1.5632 - accuracy: 0.3902
8/8 [==============================] - 4s 338ms/step - loss: 1.5965 - accuracy: 0.3486
1/1 [==============================] - 0s 419ms/step - loss: 1.6078 - accuracy: 0.2195
8/8 [==============================] - 4s 319ms/step - loss: 1.6034 - accuracy: 0.2838
1/1 [==============================] - 0s 405ms/step - loss: 1.5824 - accuracy: 0.3415
8/8 [==============================] - 6s 477ms/step - loss: 1.5977 - accuracy: 0.3108
1/1 [==============================] - 0s 382ms/step - loss: 1.5700 - accuracy: 0.4390
8/8 [==============================] - 4s 306ms/step - loss: 1.5994 - accuracy: 0.3378
1/1 [==============================] - 0s 382ms/step - loss: 1.5806 - accuracy: 0.3171
8/8 [==============================] - 5s 422ms/step - loss: 1.6006 - accuracy: 0.2405
1/1 [==============================] - 0s 399ms/step - loss: 1.5966 - accuracy: 0.2927
8/8 [==============================] - 4s 320ms/step - loss: 1.5948 - accuracy: 0.3135
1/1 [==============================] - 0s 405ms/step - loss: 1.6157 - accuracy: 0.2927
8/8 [==============================] - 5s 373ms/step - loss: 1.6012 - accuracy: 0.2892
1/1 [==============================] - 0s 377ms/step - loss: 1.5883 - accuracy: 0.3659
8/8 [==============================] - 7s 658ms/step - loss: 1.5958 - accuracy: 0.2656
1/1 [==============================] - 0s 480ms/step - loss: 1.5463 - accuracy: 0.3571
8/8 [==============================] - 7s 708ms/step - loss: 1.5916 - accuracy: 0.3054
1/1 [==============================] - 0s 495ms/step - loss: 1.5460 - accuracy: 0.3415
8/8 [==============================] - 9s 715ms/step - loss: 1.5961 - accuracy: 0.2730
1/1 [==============================] - 0s 485ms/step - loss: 1.5451 - accuracy: 0.3902
8/8 [==============================] - 10s 684ms/step - loss: 1.5845 - accuracy: 0.3162
1/1 [==============================] - 1s 503ms/step - loss: 1.6262 - accuracy: 0.2195
8/8 [==============================] - 9s 892ms/step - loss: 1.5916 - accuracy: 0.3108
1/1 [==============================] - 1s 506ms/step - loss: 1.5188 - accuracy: 0.3415
8/8 [==============================] - 7s 706ms/step - loss: 1.5885 - accuracy: 0.3216
1/1 [==============================] - 1s 529ms/step - loss: 1.4907 - accuracy: 0.4390
8/8 [==============================] - 9s 692ms/step - loss: 1.5914 - accuracy: 0.2622
1/1 [==============================] - 0s 469ms/step - loss: 1.5175 - accuracy: 0.3171
8/8 [==============================] - 10s 946ms/step - loss: 1.5845 - accuracy: 0.3189
1/1 [==============================] - 1s 551ms/step - loss: 1.5396 - accuracy: 0.2927
8/8 [==============================] - 9s 943ms/step - loss: 1.5894 - accuracy: 0.3027
1/1 [==============================] - 0s 486ms/step - loss: 1.6265 - accuracy: 0.2927
8/8 [==============================] - 9s 886ms/step - loss: 1.5899 - accuracy: 0.3000
1/1 [==============================] - 0s 479ms/step - loss: 1.5365 - accuracy: 0.3659
5/5 [==============================] - 4s 406ms/step - loss: 1.6048 - accuracy: 0.2847
best parameters for ANN:  {'batch_size': 100, 'nb_epoch': 20, 'neurons': 128}
best score for ANN: 0.3430313587188721

Best tuned parameters for Countvectorizer Method-

best parameters for ANN: {'batch_size': 100, 'nb_epoch': 20, 'neurons': 128}

best score for ANN: 0.3430313587188721

## Defining LSTM function with tuned parameters

def LSTM_Model_Tuned_CV (X_train, X_test, y_train, y_test):
  in_dim = X_train.shape[1]

  tf.random.set_seed(7)
  embedding_vecor_length = 32

  model = Sequential()
  model.add(Embedding(max_features, embedding_vecor_length, input_length=in_dim))
  model.add(SpatialDropout1D(0.2))
  model.add(LSTM(128))
  model.add(Dense(5 , activation='softmax'))

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=0, patience=3)

  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

  print(model.summary())

  history=model.fit(X_train, y_train_cat, validation_split=0.2, epochs = 20, batch_size = 100, callbacks=[early_stopping])

  # train_score = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  # test_score = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  # result_kfold_df= pd.DataFrame({'model': ['Neural Network'], 'train accuracy': [train_score], 'test accuracy': [test_score] })

  train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  result_kfold_df= pd.DataFrame({'model': ['LSTM'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })


  # print(result_kfold_df)

  hist= pd.DataFrame(history.history)

  for col in hist.columns:
    print(col)
    plt.plot(hist[col])
    plt.plot(hist[col])
    plt.title('model-'+col)
    plt.ylabel(col)
    plt.xlabel('epoch')
    plt.show()

  return result_kfold_df

Calling Tuned LSTM function

LSTM_Model_Tuned_CV(X_train_cv, X_test_cv, y_train_cv, y_test_cv)

Model: "sequential_405"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_405 (Embedding)   (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_44 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_405 (LSTM)             (None, 128)               82432     
                                                                 
 dense_405 (Dense)           (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 8s 1s/step - loss: 1.6017 - accuracy: 0.2519 - val_loss: 1.5872 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 2s 705ms/step - loss: 1.5704 - accuracy: 0.3359 - val_loss: 1.5576 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 2s 895ms/step - loss: 1.5150 - accuracy: 0.3359 - val_loss: 1.5747 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 3s 1s/step - loss: 1.5226 - accuracy: 0.3359 - val_loss: 1.5682 - val_accuracy: 0.3333
Epoch 5/20
3/3 [==============================] - 3s 843ms/step - loss: 1.4634 - accuracy: 0.3359 - val_loss: 1.5440 - val_accuracy: 0.3333
Epoch 6/20
3/3 [==============================] - 2s 804ms/step - loss: 1.4745 - accuracy: 0.3359 - val_loss: 1.5464 - val_accuracy: 0.3333
Epoch 7/20
3/3 [==============================] - 2s 779ms/step - loss: 1.4720 - accuracy: 0.3397 - val_loss: 1.5509 - val_accuracy: 0.3333
Epoch 8/20
3/3 [==============================] - 3s 938ms/step - loss: 1.4638 - accuracy: 0.3359 - val_loss: 1.5674 - val_accuracy: 0.3333
11/11 [==============================] - 1s 78ms/step
3/3 [==============================] - 0s 71ms/step
11/11 [==============================] - 1s 77ms/step
3/3 [==============================] - 0s 67ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.335366	0.337349	0.168449	0.170194

Calling Tuned LSTM function 3 times to see if results are changing.

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model_Tuned_CV(X_train_cv, X_test_cv, y_train_cv, y_test_cv)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_406"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_406 (Embedding)   (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_45 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_406 (LSTM)             (None, 128)               82432     
                                                                 
 dense_406 (Dense)           (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 5s 1s/step - loss: 1.6027 - accuracy: 0.2443 - val_loss: 1.5920 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 2s 828ms/step - loss: 1.5723 - accuracy: 0.3206 - val_loss: 1.5646 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 1s 499ms/step - loss: 1.5112 - accuracy: 0.3359 - val_loss: 1.6686 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 1s 472ms/step - loss: 1.5270 - accuracy: 0.3359 - val_loss: 1.5909 - val_accuracy: 0.3333
Epoch 5/20
3/3 [==============================] - 1s 470ms/step - loss: 1.4669 - accuracy: 0.3359 - val_loss: 1.5438 - val_accuracy: 0.3333
Epoch 6/20
3/3 [==============================] - 1s 491ms/step - loss: 1.4753 - accuracy: 0.3359 - val_loss: 1.5428 - val_accuracy: 0.3333
Epoch 7/20
3/3 [==============================] - 1s 479ms/step - loss: 1.4750 - accuracy: 0.3359 - val_loss: 1.5454 - val_accuracy: 0.3333
Epoch 8/20
3/3 [==============================] - 1s 487ms/step - loss: 1.4679 - accuracy: 0.3359 - val_loss: 1.5596 - val_accuracy: 0.3333
Epoch 9/20
3/3 [==============================] - 2s 646ms/step - loss: 1.4611 - accuracy: 0.3359 - val_loss: 1.5709 - val_accuracy: 0.3333
11/11 [==============================] - 2s 136ms/step
3/3 [==============================] - 0s 64ms/step
11/11 [==============================] - 1s 79ms/step
3/3 [==============================] - 0s 71ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_407"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_407 (Embedding)   (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_46 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_407 (LSTM)             (None, 128)               82432     
                                                                 
 dense_407 (Dense)           (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 6s 1s/step - loss: 1.6005 - accuracy: 0.2939 - val_loss: 1.5867 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 3s 893ms/step - loss: 1.5668 - accuracy: 0.3359 - val_loss: 1.5527 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 2s 545ms/step - loss: 1.5092 - accuracy: 0.3359 - val_loss: 1.6222 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 1s 485ms/step - loss: 1.5012 - accuracy: 0.3359 - val_loss: 1.5335 - val_accuracy: 0.3333
Epoch 5/20
3/3 [==============================] - 1s 477ms/step - loss: 1.4793 - accuracy: 0.3359 - val_loss: 1.5371 - val_accuracy: 0.3333
Epoch 6/20
3/3 [==============================] - 1s 470ms/step - loss: 1.4828 - accuracy: 0.3359 - val_loss: 1.5398 - val_accuracy: 0.3333
Epoch 7/20
3/3 [==============================] - 1s 473ms/step - loss: 1.4714 - accuracy: 0.3359 - val_loss: 1.5491 - val_accuracy: 0.3333
11/11 [==============================] - 1s 75ms/step
3/3 [==============================] - 0s 62ms/step
11/11 [==============================] - 1s 77ms/step
3/3 [==============================] - 0s 63ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_408"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_408 (Embedding)   (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_47 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_408 (LSTM)             (None, 128)               82432     
                                                                 
 dense_408 (Dense)           (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 4s 756ms/step - loss: 1.6008 - accuracy: 0.2939 - val_loss: 1.5883 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 2s 769ms/step - loss: 1.5682 - accuracy: 0.3397 - val_loss: 1.5589 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 2s 784ms/step - loss: 1.5083 - accuracy: 0.3359 - val_loss: 1.6548 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 1s 462ms/step - loss: 1.5124 - accuracy: 0.3359 - val_loss: 1.5488 - val_accuracy: 0.3333
Epoch 5/20
3/3 [==============================] - 1s 482ms/step - loss: 1.4664 - accuracy: 0.3359 - val_loss: 1.5371 - val_accuracy: 0.3333
Epoch 6/20
3/3 [==============================] - 1s 477ms/step - loss: 1.4751 - accuracy: 0.3359 - val_loss: 1.5395 - val_accuracy: 0.3333
Epoch 7/20
3/3 [==============================] - 1s 479ms/step - loss: 1.4697 - accuracy: 0.3359 - val_loss: 1.5495 - val_accuracy: 0.3333
Epoch 8/20
3/3 [==============================] - 1s 479ms/step - loss: 1.4655 - accuracy: 0.3359 - val_loss: 1.5761 - val_accuracy: 0.3333
11/11 [==============================] - 1s 78ms/step
3/3 [==============================] - 0s 65ms/step
11/11 [==============================] - 1s 76ms/step
3/3 [==============================] - 0s 70ms/step
loss

accuracy

val_loss

val_accuracy

Result of all runs:   model  train accuracy  test accuracy  train F1 score  test F1 score
0  LSTM        0.335366       0.337349        0.168449       0.170194
1  LSTM        0.335366       0.337349        0.168449       0.170194
2  LSTM        0.335366       0.337349        0.168449       0.170194

Result-

No change in either accuracy or F1 score for all 3 runs. It means there is result inconsistency in LSTM model.

Tuned LSTM function with CV smote dataset-

LSTM_Model_Tuned_CV(X_train_cv_smote, X_test_cv, y_train_cv_smote, y_test_cv)

Model: "sequential_409"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_409 (Embedding)   (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_48 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_409 (LSTM)             (None, 128)               82432     
                                                                 
 dense_409 (Dense)           (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 7s 907ms/step - loss: 1.5995 - accuracy: 0.2182 - val_loss: 1.7746 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 2s 472ms/step - loss: 1.5611 - accuracy: 0.2455 - val_loss: 2.6454 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 2s 466ms/step - loss: 1.5302 - accuracy: 0.2523 - val_loss: 2.3816 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 2s 453ms/step - loss: 1.5225 - accuracy: 0.2477 - val_loss: 2.3008 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 77ms/step
3/3 [==============================] - 0s 66ms/step
18/18 [==============================] - 3s 141ms/step
3/3 [==============================] - 0s 104ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.2	0.228916	0.066667	0.085282

General observation is that there is accuracy or F1 score reduction with Smote dataset in LSTM.

LSTM function with CV Full dataset-

LSTM_Model_Tuned_CV(X_train_cvfull, X_test_cvfull, y_train_cvfull, y_test_cvfull)

Model: "sequential_811"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_811 (Embedding)   (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_49 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_810 (LSTM)             (None, 128)               82432     
                                                                 
 dense_810 (Dense)           (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 10s 2s/step - loss: 1.6064 - accuracy: 0.2290 - val_loss: 1.5940 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 3s 818ms/step - loss: 1.5793 - accuracy: 0.3359 - val_loss: 1.5703 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 2s 798ms/step - loss: 1.5383 - accuracy: 0.3359 - val_loss: 1.5298 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 2s 751ms/step - loss: 1.5159 - accuracy: 0.3359 - val_loss: 1.5353 - val_accuracy: 0.3333
Epoch 5/20
3/3 [==============================] - 2s 778ms/step - loss: 1.4670 - accuracy: 0.3359 - val_loss: 1.5337 - val_accuracy: 0.3333
Epoch 6/20
3/3 [==============================] - 3s 1s/step - loss: 1.4731 - accuracy: 0.3359 - val_loss: 1.5368 - val_accuracy: 0.3333
11/11 [==============================] - 3s 153ms/step
3/3 [==============================] - 0s 94ms/step
11/11 [==============================] - 1s 128ms/step
3/3 [==============================] - 0s 77ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.335366	0.337349	0.168449	0.170194

Accuracy is remaining around 33%. F1 score is around 17%.

Running LSTM function 3 times to see if result is consistent.

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model_Tuned_CV(X_train_cvfull, X_test_cvfull, y_train_cvfull, y_test_cvfull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_812"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_812 (Embedding)   (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_50 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_811 (LSTM)             (None, 128)               82432     
                                                                 
 dense_811 (Dense)           (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 6s 1s/step - loss: 1.6019 - accuracy: 0.2557 - val_loss: 1.5906 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 2s 550ms/step - loss: 1.5730 - accuracy: 0.3359 - val_loss: 1.5622 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 2s 522ms/step - loss: 1.5159 - accuracy: 0.3359 - val_loss: 1.7041 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 2s 539ms/step - loss: 1.5182 - accuracy: 0.3359 - val_loss: 1.5382 - val_accuracy: 0.3333
Epoch 5/20
3/3 [==============================] - 2s 524ms/step - loss: 1.4801 - accuracy: 0.3359 - val_loss: 1.5426 - val_accuracy: 0.3333
Epoch 6/20
3/3 [==============================] - 2s 541ms/step - loss: 1.4864 - accuracy: 0.3321 - val_loss: 1.5467 - val_accuracy: 0.1970
Epoch 7/20
3/3 [==============================] - 3s 1s/step - loss: 1.4767 - accuracy: 0.2824 - val_loss: 1.5605 - val_accuracy: 0.2121
11/11 [==============================] - 2s 134ms/step
3/3 [==============================] - 0s 125ms/step
11/11 [==============================] - 1s 118ms/step
3/3 [==============================] - 0s 95ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_813"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_813 (Embedding)   (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_51 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_812 (LSTM)             (None, 128)               82432     
                                                                 
 dense_812 (Dense)           (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 9s 1s/step - loss: 1.6018 - accuracy: 0.2557 - val_loss: 1.5875 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 3s 1s/step - loss: 1.5685 - accuracy: 0.3359 - val_loss: 1.5550 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 4s 1s/step - loss: 1.5085 - accuracy: 0.3359 - val_loss: 1.6623 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 4s 1s/step - loss: 1.5046 - accuracy: 0.3359 - val_loss: 1.5355 - val_accuracy: 0.3333
Epoch 5/20
3/3 [==============================] - 3s 823ms/step - loss: 1.4876 - accuracy: 0.3359 - val_loss: 1.5396 - val_accuracy: 0.3333
Epoch 6/20
3/3 [==============================] - 2s 810ms/step - loss: 1.4905 - accuracy: 0.3359 - val_loss: 1.5394 - val_accuracy: 0.3333
Epoch 7/20
3/3 [==============================] - 3s 1s/step - loss: 1.4782 - accuracy: 0.3359 - val_loss: 1.5430 - val_accuracy: 0.3333
11/11 [==============================] - 1s 88ms/step
3/3 [==============================] - 0s 73ms/step
11/11 [==============================] - 1s 86ms/step
3/3 [==============================] - 0s 72ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_814"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_814 (Embedding)   (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_52 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_813 (LSTM)             (None, 128)               82432     
                                                                 
 dense_813 (Dense)           (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 11s 3s/step - loss: 1.5978 - accuracy: 0.2824 - val_loss: 1.5835 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 4s 1s/step - loss: 1.5592 - accuracy: 0.3359 - val_loss: 1.5519 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 5s 2s/step - loss: 1.4961 - accuracy: 0.3359 - val_loss: 1.6612 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 3s 1s/step - loss: 1.5007 - accuracy: 0.3359 - val_loss: 1.5480 - val_accuracy: 0.3333
Epoch 5/20
3/3 [==============================] - 3s 1s/step - loss: 1.4654 - accuracy: 0.3359 - val_loss: 1.5385 - val_accuracy: 0.3333
Epoch 6/20
3/3 [==============================] - 3s 1s/step - loss: 1.4756 - accuracy: 0.3359 - val_loss: 1.5394 - val_accuracy: 0.3333
Epoch 7/20
3/3 [==============================] - 5s 2s/step - loss: 1.4744 - accuracy: 0.3359 - val_loss: 1.5416 - val_accuracy: 0.3333
Epoch 8/20
3/3 [==============================] - 4s 1s/step - loss: 1.4670 - accuracy: 0.3359 - val_loss: 1.5526 - val_accuracy: 0.3333
11/11 [==============================] - 4s 181ms/step
3/3 [==============================] - 1s 140ms/step
11/11 [==============================] - 2s 202ms/step
3/3 [==============================] - 1s 145ms/step
loss

accuracy

val_loss

val_accuracy

Result of all runs:   model  train accuracy  test accuracy  train F1 score  test F1 score
0  LSTM        0.335366       0.337349        0.168449       0.170194
1  LSTM        0.335366       0.337349        0.168449       0.170194
2  LSTM        0.259146       0.253012        0.106670       0.102178

LSTM tuned function with CV full smote dataset-

LSTM_Model_Tuned_CV(X_train_cvfull_smote, X_test_cvfull, y_train_cvfull_smote, y_test_cvfull)

Model: "sequential_815"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_815 (Embedding)   (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_53 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_814 (LSTM)             (None, 128)               82432     
                                                                 
 dense_814 (Dense)           (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 12s 1s/step - loss: 1.5987 - accuracy: 0.2409 - val_loss: 1.7819 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 4s 726ms/step - loss: 1.5567 - accuracy: 0.2500 - val_loss: 3.1241 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 4s 735ms/step - loss: 1.5379 - accuracy: 0.2455 - val_loss: 2.3688 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 5s 1s/step - loss: 1.5243 - accuracy: 0.2386 - val_loss: 2.2779 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 118ms/step
3/3 [==============================] - 0s 115ms/step
18/18 [==============================] - 2s 110ms/step
3/3 [==============================] - 0s 87ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.2	0.108434	0.066667	0.021215

Running Tuned LSTM function 3 times to see if result is consistent-

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(5):
  result=LSTM_Model_Tuned_CV(X_train_cvfull_smote, X_test_cvfull, y_train_cvfull_smote, y_test_cvfull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_816"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_816 (Embedding)   (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_54 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_815 (LSTM)             (None, 128)               82432     
                                                                 
 dense_815 (Dense)           (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 7s 882ms/step - loss: 1.6016 - accuracy: 0.1977 - val_loss: 1.7561 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 3s 506ms/step - loss: 1.5671 - accuracy: 0.2523 - val_loss: 2.4586 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 3s 514ms/step - loss: 1.5282 - accuracy: 0.2455 - val_loss: 2.3878 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 3s 513ms/step - loss: 1.5205 - accuracy: 0.2523 - val_loss: 2.4558 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 151ms/step
3/3 [==============================] - 0s 72ms/step
18/18 [==============================] - 2s 85ms/step
3/3 [==============================] - 0s 135ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_817"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_817 (Embedding)   (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_55 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_816 (LSTM)             (None, 128)               82432     
                                                                 
 dense_816 (Dense)           (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 12s 1s/step - loss: 1.5999 - accuracy: 0.2341 - val_loss: 1.7759 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 6s 1s/step - loss: 1.5589 - accuracy: 0.2477 - val_loss: 3.3161 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 6s 1s/step - loss: 1.5405 - accuracy: 0.2932 - val_loss: 2.3210 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 5s 1s/step - loss: 1.5263 - accuracy: 0.2432 - val_loss: 2.2336 - val_accuracy: 0.0000e+00
18/18 [==============================] - 6s 211ms/step
3/3 [==============================] - 1s 168ms/step
18/18 [==============================] - 3s 174ms/step
3/3 [==============================] - 1s 307ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_818"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_818 (Embedding)   (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_56 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_817 (LSTM)             (None, 128)               82432     
                                                                 
 dense_817 (Dense)           (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 11s 1s/step - loss: 1.5946 - accuracy: 0.1795 - val_loss: 1.8462 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 5s 955ms/step - loss: 1.5531 - accuracy: 0.2432 - val_loss: 2.5941 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 6s 1s/step - loss: 1.5162 - accuracy: 0.2591 - val_loss: 2.6188 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 5s 870ms/step - loss: 1.5182 - accuracy: 0.2500 - val_loss: 2.5638 - val_accuracy: 0.0000e+00
18/18 [==============================] - 5s 172ms/step
3/3 [==============================] - 1s 205ms/step
18/18 [==============================] - 5s 257ms/step
3/3 [==============================] - 0s 83ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_819"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_819 (Embedding)   (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_57 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_818 (LSTM)             (None, 128)               82432     
                                                                 
 dense_818 (Dense)           (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 14s 1s/step - loss: 1.6018 - accuracy: 0.2341 - val_loss: 1.7453 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 4s 798ms/step - loss: 1.5659 - accuracy: 0.2568 - val_loss: 2.8062 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 6s 1s/step - loss: 1.5419 - accuracy: 0.2773 - val_loss: 2.3826 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 4s 897ms/step - loss: 1.5252 - accuracy: 0.2545 - val_loss: 2.2830 - val_accuracy: 0.0000e+00
18/18 [==============================] - 4s 174ms/step
3/3 [==============================] - 1s 199ms/step
18/18 [==============================] - 4s 191ms/step
3/3 [==============================] - 1s 166ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_820"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_820 (Embedding)   (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_58 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_819 (LSTM)             (None, 128)               82432     
                                                                 
 dense_819 (Dense)           (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 10s 936ms/step - loss: 1.5959 - accuracy: 0.2477 - val_loss: 1.8127 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 5s 1s/step - loss: 1.5563 - accuracy: 0.2477 - val_loss: 2.7269 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 6s 1s/step - loss: 1.5272 - accuracy: 0.2705 - val_loss: 2.4346 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 5s 1s/step - loss: 1.5218 - accuracy: 0.2523 - val_loss: 2.3909 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 121ms/step
3/3 [==============================] - 0s 101ms/step
18/18 [==============================] - 2s 135ms/step
3/3 [==============================] - 0s 109ms/step
loss

accuracy

val_loss

val_accuracy

Result of all runs:   model  train accuracy  test accuracy  train F1 score  test F1 score
0  LSTM             0.2       0.228916        0.066667       0.085282
1  LSTM             0.2       0.228916        0.066768       0.085282
2  LSTM             0.2       0.228916        0.066667       0.085282
3  LSTM             0.2       0.108434        0.066667       0.021215
4  LSTM             0.2       0.228916        0.066667       0.085282

Observations-

1. Accuracy is remaining around 33%. F1 score is around 17%.
2. Smote results are lower compared to non-smote.

TFIDF Dataset

Tuned_LSTM(X_tfidf_df, y_tfidf_df)

4/4 [==============================] - 2s 88ms/step - loss: 1.6109 - accuracy: 0.1084
1/1 [==============================] - 0s 309ms/step - loss: 1.6054 - accuracy: 0.0714
4/4 [==============================] - 2s 88ms/step - loss: 1.6071 - accuracy: 0.2378
1/1 [==============================] - 0s 311ms/step - loss: 1.6050 - accuracy: 0.1707
4/4 [==============================] - 2s 92ms/step - loss: 1.6042 - accuracy: 0.3297
1/1 [==============================] - 0s 314ms/step - loss: 1.5993 - accuracy: 0.2195
4/4 [==============================] - 3s 147ms/step - loss: 1.6149 - accuracy: 0.0649
1/1 [==============================] - 1s 641ms/step - loss: 1.6094 - accuracy: 0.2683
4/4 [==============================] - 3s 89ms/step - loss: 1.6070 - accuracy: 0.1514
1/1 [==============================] - 0s 313ms/step - loss: 1.6009 - accuracy: 0.2439
4/4 [==============================] - 2s 89ms/step - loss: 1.6080 - accuracy: 0.0757
1/1 [==============================] - 0s 302ms/step - loss: 1.5992 - accuracy: 0.4390
4/4 [==============================] - 2s 90ms/step - loss: 1.5993 - accuracy: 0.3135
1/1 [==============================] - 0s 304ms/step - loss: 1.5857 - accuracy: 0.3171
4/4 [==============================] - 3s 88ms/step - loss: 1.6073 - accuracy: 0.2405
1/1 [==============================] - 0s 467ms/step - loss: 1.6072 - accuracy: 0.1463
4/4 [==============================] - 3s 91ms/step - loss: 1.6067 - accuracy: 0.2432
1/1 [==============================] - 0s 321ms/step - loss: 1.6104 - accuracy: 0.1220
4/4 [==============================] - 2s 88ms/step - loss: 1.6163 - accuracy: 0.0703
1/1 [==============================] - 0s 305ms/step - loss: 1.6109 - accuracy: 0.0732
4/4 [==============================] - 2s 96ms/step - loss: 1.6063 - accuracy: 0.2575
1/1 [==============================] - 0s 319ms/step - loss: 1.6007 - accuracy: 0.2619
4/4 [==============================] - 2s 90ms/step - loss: 1.6013 - accuracy: 0.3351
1/1 [==============================] - 0s 324ms/step - loss: 1.5895 - accuracy: 0.3415
4/4 [==============================] - 3s 141ms/step - loss: 1.6096 - accuracy: 0.2162
1/1 [==============================] - 0s 481ms/step - loss: 1.6000 - accuracy: 0.3902
4/4 [==============================] - 2s 93ms/step - loss: 1.6018 - accuracy: 0.3486
1/1 [==============================] - 0s 316ms/step - loss: 1.6052 - accuracy: 0.2195
4/4 [==============================] - 2s 95ms/step - loss: 1.5995 - accuracy: 0.3351
1/1 [==============================] - 0s 310ms/step - loss: 1.5892 - accuracy: 0.3415
4/4 [==============================] - 2s 90ms/step - loss: 1.6037 - accuracy: 0.2649
1/1 [==============================] - 0s 299ms/step - loss: 1.5974 - accuracy: 0.1951
4/4 [==============================] - 2s 109ms/step - loss: 1.6066 - accuracy: 0.2216
1/1 [==============================] - 0s 439ms/step - loss: 1.5949 - accuracy: 0.3171
4/4 [==============================] - 3s 91ms/step - loss: 1.6050 - accuracy: 0.3081
1/1 [==============================] - 0s 300ms/step - loss: 1.6033 - accuracy: 0.1463
4/4 [==============================] - 2s 89ms/step - loss: 1.5998 - accuracy: 0.2676
1/1 [==============================] - 0s 297ms/step - loss: 1.6098 - accuracy: 0.1707
4/4 [==============================] - 2s 91ms/step - loss: 1.6053 - accuracy: 0.2973
1/1 [==============================] - 1s 1s/step - loss: 1.5911 - accuracy: 0.3659
4/4 [==============================] - 2s 114ms/step - loss: 1.6047 - accuracy: 0.2818
1/1 [==============================] - 0s 421ms/step - loss: 1.5996 - accuracy: 0.3571
4/4 [==============================] - 3s 139ms/step - loss: 1.6062 - accuracy: 0.2892
1/1 [==============================] - 0s 335ms/step - loss: 1.6013 - accuracy: 0.3415
4/4 [==============================] - 2s 108ms/step - loss: 1.6087 - accuracy: 0.1108
1/1 [==============================] - 0s 335ms/step - loss: 1.6052 - accuracy: 0.0488
4/4 [==============================] - 2s 107ms/step - loss: 1.6117 - accuracy: 0.2568
1/1 [==============================] - 0s 329ms/step - loss: 1.6073 - accuracy: 0.2683
4/4 [==============================] - 2s 114ms/step - loss: 1.6111 - accuracy: 0.1568
1/1 [==============================] - 0s 312ms/step - loss: 1.6075 - accuracy: 0.3415
4/4 [==============================] - 3s 174ms/step - loss: 1.6067 - accuracy: 0.2108
1/1 [==============================] - 0s 492ms/step - loss: 1.6011 - accuracy: 0.1951
4/4 [==============================] - 2s 111ms/step - loss: 1.6125 - accuracy: 0.1459
1/1 [==============================] - 0s 309ms/step - loss: 1.6038 - accuracy: 0.3171
4/4 [==============================] - 2s 116ms/step - loss: 1.6017 - accuracy: 0.3405
1/1 [==============================] - 0s 313ms/step - loss: 1.6023 - accuracy: 0.2927
4/4 [==============================] - 2s 110ms/step - loss: 1.6015 - accuracy: 0.2432
1/1 [==============================] - 0s 312ms/step - loss: 1.6141 - accuracy: 0.1220
4/4 [==============================] - 2s 173ms/step - loss: 1.6021 - accuracy: 0.3324
1/1 [==============================] - 0s 430ms/step - loss: 1.5963 - accuracy: 0.3659
4/4 [==============================] - 3s 202ms/step - loss: 1.5973 - accuracy: 0.3333
1/1 [==============================] - 0s 318ms/step - loss: 1.5834 - accuracy: 0.3571
4/4 [==============================] - 2s 211ms/step - loss: 1.6058 - accuracy: 0.2541
1/1 [==============================] - 1s 1s/step - loss: 1.6026 - accuracy: 0.3415
4/4 [==============================] - 3s 211ms/step - loss: 1.6104 - accuracy: 0.2514
1/1 [==============================] - 1s 502ms/step - loss: 1.5991 - accuracy: 0.3171
4/4 [==============================] - 3s 203ms/step - loss: 1.6081 - accuracy: 0.1297
1/1 [==============================] - 0s 348ms/step - loss: 1.6065 - accuracy: 0.2195
4/4 [==============================] - 2s 207ms/step - loss: 1.5982 - accuracy: 0.3351
1/1 [==============================] - 0s 327ms/step - loss: 1.5883 - accuracy: 0.3415
4/4 [==============================] - 2s 209ms/step - loss: 1.6037 - accuracy: 0.3081
1/1 [==============================] - 0s 347ms/step - loss: 1.5927 - accuracy: 0.4390
4/4 [==============================] - 3s 381ms/step - loss: 1.6030 - accuracy: 0.3378
1/1 [==============================] - 1s 511ms/step - loss: 1.5955 - accuracy: 0.3171
4/4 [==============================] - 2s 209ms/step - loss: 1.6011 - accuracy: 0.2405
1/1 [==============================] - 0s 331ms/step - loss: 1.6058 - accuracy: 0.1463
4/4 [==============================] - 2s 200ms/step - loss: 1.6015 - accuracy: 0.2865
1/1 [==============================] - 0s 325ms/step - loss: 1.6100 - accuracy: 0.1707
4/4 [==============================] - 2s 204ms/step - loss: 1.6045 - accuracy: 0.3324
1/1 [==============================] - 0s 337ms/step - loss: 1.5954 - accuracy: 0.3659
4/4 [==============================] - 4s 424ms/step - loss: 1.6024 - accuracy: 0.3089
1/1 [==============================] - 0s 379ms/step - loss: 1.5875 - accuracy: 0.3571
4/4 [==============================] - 3s 418ms/step - loss: 1.6036 - accuracy: 0.3108
1/1 [==============================] - 0s 388ms/step - loss: 1.5981 - accuracy: 0.3415
4/4 [==============================] - 3s 435ms/step - loss: 1.6021 - accuracy: 0.2946
1/1 [==============================] - 1s 591ms/step - loss: 1.5854 - accuracy: 0.3902
4/4 [==============================] - 4s 417ms/step - loss: 1.6054 - accuracy: 0.2946
1/1 [==============================] - 0s 369ms/step - loss: 1.6091 - accuracy: 0.2195
4/4 [==============================] - 3s 416ms/step - loss: 1.6054 - accuracy: 0.2162
1/1 [==============================] - 0s 382ms/step - loss: 1.5917 - accuracy: 0.3415
4/4 [==============================] - 5s 781ms/step - loss: 1.6069 - accuracy: 0.1838
1/1 [==============================] - 0s 401ms/step - loss: 1.5976 - accuracy: 0.4390
4/4 [==============================] - 3s 420ms/step - loss: 1.6048 - accuracy: 0.3378
1/1 [==============================] - 0s 388ms/step - loss: 1.5967 - accuracy: 0.3171
4/4 [==============================] - 3s 419ms/step - loss: 1.6028 - accuracy: 0.3081
1/1 [==============================] - 0s 385ms/step - loss: 1.5963 - accuracy: 0.2927
4/4 [==============================] - 5s 694ms/step - loss: 1.6067 - accuracy: 0.2973
1/1 [==============================] - 1s 572ms/step - loss: 1.6080 - accuracy: 0.2927
4/4 [==============================] - 4s 425ms/step - loss: 1.6055 - accuracy: 0.2622
1/1 [==============================] - 0s 395ms/step - loss: 1.5979 - accuracy: 0.3659
4/4 [==============================] - 7s 1s/step - loss: 1.5996 - accuracy: 0.3089
1/1 [==============================] - 0s 472ms/step - loss: 1.5762 - accuracy: 0.3571
4/4 [==============================] - 5s 957ms/step - loss: 1.6009 - accuracy: 0.3351
1/1 [==============================] - 0s 476ms/step - loss: 1.5873 - accuracy: 0.3415
4/4 [==============================] - 7s 964ms/step - loss: 1.5985 - accuracy: 0.3297
1/1 [==============================] - 1s 505ms/step - loss: 1.5718 - accuracy: 0.3902
4/4 [==============================] - 5s 960ms/step - loss: 1.6046 - accuracy: 0.2027
1/1 [==============================] - 1s 501ms/step - loss: 1.6119 - accuracy: 0.2195
4/4 [==============================] - 6s 972ms/step - loss: 1.6061 - accuracy: 0.2595
1/1 [==============================] - 0s 493ms/step - loss: 1.5930 - accuracy: 0.3415
4/4 [==============================] - 8s 1s/step - loss: 1.6041 - accuracy: 0.2270
1/1 [==============================] - 1s 842ms/step - loss: 1.5892 - accuracy: 0.2683
4/4 [==============================] - 6s 967ms/step - loss: 1.6049 - accuracy: 0.2351
1/1 [==============================] - 0s 484ms/step - loss: 1.5889 - accuracy: 0.3171
4/4 [==============================] - 7s 1s/step - loss: 1.5994 - accuracy: 0.2459
1/1 [==============================] - 0s 489ms/step - loss: 1.5927 - accuracy: 0.2927
4/4 [==============================] - 7s 1s/step - loss: 1.6009 - accuracy: 0.3405
1/1 [==============================] - 1s 690ms/step - loss: 1.6082 - accuracy: 0.2927
4/4 [==============================] - 6s 991ms/step - loss: 1.6021 - accuracy: 0.2703
1/1 [==============================] - 0s 476ms/step - loss: 1.5897 - accuracy: 0.1463
4/4 [==============================] - 2s 112ms/step - loss: 1.6097 - accuracy: 0.1870
1/1 [==============================] - 0s 433ms/step - loss: 1.6075 - accuracy: 0.2381
4/4 [==============================] - 3s 89ms/step - loss: 1.6270 - accuracy: 0.1135
1/1 [==============================] - 0s 307ms/step - loss: 1.6233 - accuracy: 0.0244
4/4 [==============================] - 2s 89ms/step - loss: 1.6133 - accuracy: 0.0757
1/1 [==============================] - 0s 309ms/step - loss: 1.6086 - accuracy: 0.2195
4/4 [==============================] - 2s 90ms/step - loss: 1.6006 - accuracy: 0.2432
1/1 [==============================] - 0s 321ms/step - loss: 1.6105 - accuracy: 0.2195
4/4 [==============================] - 2s 94ms/step - loss: 1.6065 - accuracy: 0.3351
1/1 [==============================] - 0s 314ms/step - loss: 1.6003 - accuracy: 0.3415
4/4 [==============================] - 3s 143ms/step - loss: 1.6026 - accuracy: 0.3243
1/1 [==============================] - 0s 485ms/step - loss: 1.5931 - accuracy: 0.4390
4/4 [==============================] - 2s 90ms/step - loss: 1.6028 - accuracy: 0.2216
1/1 [==============================] - 0s 305ms/step - loss: 1.5902 - accuracy: 0.3171
4/4 [==============================] - 2s 86ms/step - loss: 1.6102 - accuracy: 0.2919
1/1 [==============================] - 1s 1s/step - loss: 1.6093 - accuracy: 0.2927
4/4 [==============================] - 2s 95ms/step - loss: 1.6057 - accuracy: 0.3405
1/1 [==============================] - 0s 343ms/step - loss: 1.6101 - accuracy: 0.2927
4/4 [==============================] - 3s 127ms/step - loss: 1.6061 - accuracy: 0.2189
1/1 [==============================] - 0s 480ms/step - loss: 1.6065 - accuracy: 0.3659
4/4 [==============================] - 2s 93ms/step - loss: 1.6018 - accuracy: 0.2575
1/1 [==============================] - 0s 308ms/step - loss: 1.5954 - accuracy: 0.2619
4/4 [==============================] - 2s 97ms/step - loss: 1.6085 - accuracy: 0.2459
1/1 [==============================] - 0s 329ms/step - loss: 1.6003 - accuracy: 0.3659
4/4 [==============================] - 2s 90ms/step - loss: 1.6087 - accuracy: 0.2324
1/1 [==============================] - 0s 304ms/step - loss: 1.6024 - accuracy: 0.2195
4/4 [==============================] - 2s 140ms/step - loss: 1.6078 - accuracy: 0.2757
1/1 [==============================] - 0s 426ms/step - loss: 1.6110 - accuracy: 0.2195
4/4 [==============================] - 2s 91ms/step - loss: 1.6050 - accuracy: 0.3351
1/1 [==============================] - 0s 319ms/step - loss: 1.5996 - accuracy: 0.3415
4/4 [==============================] - 2s 98ms/step - loss: 1.6047 - accuracy: 0.3243
1/1 [==============================] - 0s 308ms/step - loss: 1.5982 - accuracy: 0.4390
4/4 [==============================] - 2s 90ms/step - loss: 1.6105 - accuracy: 0.1676
1/1 [==============================] - 0s 323ms/step - loss: 1.6076 - accuracy: 0.3171
4/4 [==============================] - 2s 93ms/step - loss: 1.6098 - accuracy: 0.2027
1/1 [==============================] - 0s 301ms/step - loss: 1.6086 - accuracy: 0.1463
4/4 [==============================] - 3s 141ms/step - loss: 1.6012 - accuracy: 0.2973
1/1 [==============================] - 0s 477ms/step - loss: 1.6076 - accuracy: 0.2927
4/4 [==============================] - 2s 94ms/step - loss: 1.6060 - accuracy: 0.3324
1/1 [==============================] - 0s 294ms/step - loss: 1.5996 - accuracy: 0.3659
4/4 [==============================] - 2s 114ms/step - loss: 1.6059 - accuracy: 0.3333
1/1 [==============================] - 0s 314ms/step - loss: 1.5984 - accuracy: 0.3571
4/4 [==============================] - 2s 118ms/step - loss: 1.6030 - accuracy: 0.3351
1/1 [==============================] - 0s 327ms/step - loss: 1.5972 - accuracy: 0.3415
4/4 [==============================] - 3s 171ms/step - loss: 1.6029 - accuracy: 0.2324
1/1 [==============================] - 0s 483ms/step - loss: 1.5954 - accuracy: 0.2195
4/4 [==============================] - 2s 109ms/step - loss: 1.6110 - accuracy: 0.2027
1/1 [==============================] - 0s 339ms/step - loss: 1.6113 - accuracy: 0.1220
4/4 [==============================] - 2s 110ms/step - loss: 1.6024 - accuracy: 0.2595
1/1 [==============================] - 0s 321ms/step - loss: 1.5973 - accuracy: 0.2439
4/4 [==============================] - 2s 120ms/step - loss: 1.6033 - accuracy: 0.2649
1/1 [==============================] - 0s 316ms/step - loss: 1.5980 - accuracy: 0.1951
4/4 [==============================] - 3s 197ms/step - loss: 1.6082 - accuracy: 0.2297
1/1 [==============================] - 1s 544ms/step - loss: 1.6015 - accuracy: 0.3171
4/4 [==============================] - 3s 114ms/step - loss: 1.6040 - accuracy: 0.3243
1/1 [==============================] - 0s 319ms/step - loss: 1.6002 - accuracy: 0.2927
4/4 [==============================] - 2s 110ms/step - loss: 1.6050 - accuracy: 0.3405
1/1 [==============================] - 0s 323ms/step - loss: 1.6082 - accuracy: 0.2927
4/4 [==============================] - 2s 115ms/step - loss: 1.6081 - accuracy: 0.2189
1/1 [==============================] - 0s 304ms/step - loss: 1.6028 - accuracy: 0.1463
4/4 [==============================] - 2s 205ms/step - loss: 1.5991 - accuracy: 0.3333
1/1 [==============================] - 0s 418ms/step - loss: 1.5861 - accuracy: 0.3571
4/4 [==============================] - 3s 211ms/step - loss: 1.6045 - accuracy: 0.2378
1/1 [==============================] - 0s 349ms/step - loss: 1.5960 - accuracy: 0.1707
4/4 [==============================] - 2s 202ms/step - loss: 1.6073 - accuracy: 0.1108
1/1 [==============================] - 0s 354ms/step - loss: 1.5995 - accuracy: 0.3902
4/4 [==============================] - 2s 202ms/step - loss: 1.6106 - accuracy: 0.2432
1/1 [==============================] - 0s 333ms/step - loss: 1.6111 - accuracy: 0.2195
4/4 [==============================] - 3s 361ms/step - loss: 1.6075 - accuracy: 0.2216
1/1 [==============================] - 0s 332ms/step - loss: 1.5986 - accuracy: 0.3415
4/4 [==============================] - 2s 200ms/step - loss: 1.6049 - accuracy: 0.2649
1/1 [==============================] - 0s 334ms/step - loss: 1.5967 - accuracy: 0.1951
4/4 [==============================] - 2s 203ms/step - loss: 1.6056 - accuracy: 0.2622
1/1 [==============================] - 0s 334ms/step - loss: 1.5961 - accuracy: 0.3171
4/4 [==============================] - 2s 211ms/step - loss: 1.6083 - accuracy: 0.1811
1/1 [==============================] - 0s 462ms/step - loss: 1.6051 - accuracy: 0.1463
4/4 [==============================] - 3s 207ms/step - loss: 1.6059 - accuracy: 0.2973
1/1 [==============================] - 0s 341ms/step - loss: 1.6073 - accuracy: 0.2927
4/4 [==============================] - 2s 205ms/step - loss: 1.6058 - accuracy: 0.2189
1/1 [==============================] - 0s 345ms/step - loss: 1.5989 - accuracy: 0.3659
4/4 [==============================] - 3s 425ms/step - loss: 1.6061 - accuracy: 0.2304
1/1 [==============================] - 0s 400ms/step - loss: 1.5955 - accuracy: 0.3571
4/4 [==============================] - 5s 560ms/step - loss: 1.6047 - accuracy: 0.2703
1/1 [==============================] - 0s 390ms/step - loss: 1.5967 - accuracy: 0.3415
4/4 [==============================] - 3s 427ms/step - loss: 1.6047 - accuracy: 0.2946
1/1 [==============================] - 0s 381ms/step - loss: 1.5936 - accuracy: 0.3902
4/4 [==============================] - 4s 427ms/step - loss: 1.6033 - accuracy: 0.2946
1/1 [==============================] - 1s 638ms/step - loss: 1.6047 - accuracy: 0.2195
4/4 [==============================] - 4s 426ms/step - loss: 1.6017 - accuracy: 0.2378
1/1 [==============================] - 0s 384ms/step - loss: 1.5791 - accuracy: 0.3415
4/4 [==============================] - 3s 426ms/step - loss: 1.6089 - accuracy: 0.2784
1/1 [==============================] - 0s 376ms/step - loss: 1.5972 - accuracy: 0.4390
4/4 [==============================] - 4s 644ms/step - loss: 1.6037 - accuracy: 0.2216
1/1 [==============================] - 1s 610ms/step - loss: 1.5905 - accuracy: 0.3171
4/4 [==============================] - 3s 419ms/step - loss: 1.6098 - accuracy: 0.1324
1/1 [==============================] - 0s 393ms/step - loss: 1.6027 - accuracy: 0.3415
4/4 [==============================] - 3s 422ms/step - loss: 1.6047 - accuracy: 0.3405
1/1 [==============================] - 0s 381ms/step - loss: 1.6058 - accuracy: 0.2927
4/4 [==============================] - 4s 706ms/step - loss: 1.6001 - accuracy: 0.3324
1/1 [==============================] - 1s 574ms/step - loss: 1.5858 - accuracy: 0.3659
4/4 [==============================] - 6s 966ms/step - loss: 1.6042 - accuracy: 0.2547
1/1 [==============================] - 0s 491ms/step - loss: 1.5874 - accuracy: 0.3571
4/4 [==============================] - 7s 1s/step - loss: 1.6023 - accuracy: 0.3351
1/1 [==============================] - 1s 661ms/step - loss: 1.5900 - accuracy: 0.3415
4/4 [==============================] - 6s 980ms/step - loss: 1.6016 - accuracy: 0.2946
1/1 [==============================] - 0s 495ms/step - loss: 1.5792 - accuracy: 0.3902
4/4 [==============================] - 7s 1s/step - loss: 1.6010 - accuracy: 0.3486
1/1 [==============================] - 0s 475ms/step - loss: 1.6082 - accuracy: 0.2195
4/4 [==============================] - 6s 965ms/step - loss: 1.6033 - accuracy: 0.2757
1/1 [==============================] - 0s 470ms/step - loss: 1.5857 - accuracy: 0.3415
4/4 [==============================] - 9s 974ms/step - loss: 1.6031 - accuracy: 0.3243
1/1 [==============================] - 1s 522ms/step - loss: 1.5848 - accuracy: 0.4390
4/4 [==============================] - 6s 1s/step - loss: 1.6027 - accuracy: 0.3378
1/1 [==============================] - 1s 822ms/step - loss: 1.5861 - accuracy: 0.3171
4/4 [==============================] - 6s 982ms/step - loss: 1.6017 - accuracy: 0.2027
1/1 [==============================] - 0s 497ms/step - loss: 1.5973 - accuracy: 0.2927
4/4 [==============================] - 7s 1s/step - loss: 1.6024 - accuracy: 0.3405
1/1 [==============================] - 0s 483ms/step - loss: 1.6085 - accuracy: 0.2927
4/4 [==============================] - 7s 1s/step - loss: 1.6045 - accuracy: 0.2757
1/1 [==============================] - 1s 821ms/step - loss: 1.5933 - accuracy: 0.3659
19/19 [==============================] - 4s 120ms/step - loss: 1.6062 - accuracy: 0.2520
3/3 [==============================] - 0s 16ms/step - loss: 1.6021 - accuracy: 0.3571
19/19 [==============================] - 3s 76ms/step - loss: 1.5881 - accuracy: 0.3351
3/3 [==============================] - 0s 15ms/step - loss: 1.5688 - accuracy: 0.3415
19/19 [==============================] - 3s 76ms/step - loss: 1.5932 - accuracy: 0.2730
3/3 [==============================] - 0s 15ms/step - loss: 1.5637 - accuracy: 0.3902
19/19 [==============================] - 4s 111ms/step - loss: 1.5981 - accuracy: 0.3162
3/3 [==============================] - 1s 23ms/step - loss: 1.6087 - accuracy: 0.2195
19/19 [==============================] - 3s 76ms/step - loss: 1.5967 - accuracy: 0.3351
3/3 [==============================] - 0s 16ms/step - loss: 1.5738 - accuracy: 0.3415
19/19 [==============================] - 3s 77ms/step - loss: 1.5985 - accuracy: 0.2297
3/3 [==============================] - 0s 15ms/step - loss: 1.5759 - accuracy: 0.4390
19/19 [==============================] - 4s 113ms/step - loss: 1.6050 - accuracy: 0.2568
3/3 [==============================] - 0s 15ms/step - loss: 1.5858 - accuracy: 0.2683
19/19 [==============================] - 3s 75ms/step - loss: 1.6048 - accuracy: 0.2297
3/3 [==============================] - 0s 16ms/step - loss: 1.6050 - accuracy: 0.1463
19/19 [==============================] - 3s 77ms/step - loss: 1.5985 - accuracy: 0.3378
3/3 [==============================] - 0s 16ms/step - loss: 1.6121 - accuracy: 0.2927
19/19 [==============================] - 4s 114ms/step - loss: 1.6089 - accuracy: 0.1351
3/3 [==============================] - 0s 16ms/step - loss: 1.5999 - accuracy: 0.3415
19/19 [==============================] - 3s 80ms/step - loss: 1.6038 - accuracy: 0.2737
3/3 [==============================] - 0s 16ms/step - loss: 1.5950 - accuracy: 0.3571
19/19 [==============================] - 3s 81ms/step - loss: 1.6019 - accuracy: 0.2378
3/3 [==============================] - 0s 16ms/step - loss: 1.5995 - accuracy: 0.1707
19/19 [==============================] - 4s 115ms/step - loss: 1.6046 - accuracy: 0.2865
3/3 [==============================] - 0s 16ms/step - loss: 1.5929 - accuracy: 0.3902
19/19 [==============================] - 3s 81ms/step - loss: 1.5981 - accuracy: 0.2432
3/3 [==============================] - 0s 14ms/step - loss: 1.6067 - accuracy: 0.1220
19/19 [==============================] - 3s 80ms/step - loss: 1.5889 - accuracy: 0.3351
3/3 [==============================] - 0s 16ms/step - loss: 1.5461 - accuracy: 0.3415
19/19 [==============================] - 4s 113ms/step - loss: 1.6033 - accuracy: 0.2946
3/3 [==============================] - 0s 15ms/step - loss: 1.5848 - accuracy: 0.4390
19/19 [==============================] - 3s 80ms/step - loss: 1.5958 - accuracy: 0.3189
3/3 [==============================] - 0s 16ms/step - loss: 1.5756 - accuracy: 0.3171
19/19 [==============================] - 3s 79ms/step - loss: 1.5916 - accuracy: 0.3405
3/3 [==============================] - 0s 15ms/step - loss: 1.5848 - accuracy: 0.2927
19/19 [==============================] - 4s 111ms/step - loss: 1.5934 - accuracy: 0.2405
3/3 [==============================] - 0s 16ms/step - loss: 1.6136 - accuracy: 0.1220
19/19 [==============================] - 3s 80ms/step - loss: 1.6059 - accuracy: 0.2703
3/3 [==============================] - 1s 16ms/step - loss: 1.6021 - accuracy: 0.1463
19/19 [==============================] - 3s 88ms/step - loss: 1.5958 - accuracy: 0.2412
3/3 [==============================] - 0s 17ms/step - loss: 1.5699 - accuracy: 0.3571
19/19 [==============================] - 4s 111ms/step - loss: 1.5872 - accuracy: 0.3351
3/3 [==============================] - 0s 17ms/step - loss: 1.5700 - accuracy: 0.3415
19/19 [==============================] - 3s 86ms/step - loss: 1.5865 - accuracy: 0.2351
3/3 [==============================] - 0s 16ms/step - loss: 1.5430 - accuracy: 0.2195
19/19 [==============================] - 3s 86ms/step - loss: 1.5959 - accuracy: 0.3054
3/3 [==============================] - 0s 17ms/step - loss: 1.6177 - accuracy: 0.2195
19/19 [==============================] - 4s 124ms/step - loss: 1.6018 - accuracy: 0.3108
3/3 [==============================] - 0s 17ms/step - loss: 1.5866 - accuracy: 0.3415
19/19 [==============================] - 3s 87ms/step - loss: 1.5981 - accuracy: 0.1973
3/3 [==============================] - 0s 19ms/step - loss: 1.5688 - accuracy: 0.4390
19/19 [==============================] - 3s 87ms/step - loss: 1.5924 - accuracy: 0.3189
3/3 [==============================] - 0s 17ms/step - loss: 1.5535 - accuracy: 0.3171
19/19 [==============================] - 4s 114ms/step - loss: 1.6083 - accuracy: 0.1838
3/3 [==============================] - 0s 17ms/step - loss: 1.6044 - accuracy: 0.2927
19/19 [==============================] - 3s 87ms/step - loss: 1.6026 - accuracy: 0.2676
3/3 [==============================] - 0s 15ms/step - loss: 1.6112 - accuracy: 0.1707
19/19 [==============================] - 3s 86ms/step - loss: 1.5887 - accuracy: 0.3297
3/3 [==============================] - 0s 18ms/step - loss: 1.5436 - accuracy: 0.3659
19/19 [==============================] - 5s 123ms/step - loss: 1.5819 - accuracy: 0.2304
3/3 [==============================] - 0s 20ms/step - loss: 1.5242 - accuracy: 0.2381
19/19 [==============================] - 3s 101ms/step - loss: 1.5820 - accuracy: 0.3135
3/3 [==============================] - 0s 20ms/step - loss: 1.5354 - accuracy: 0.3415
19/19 [==============================] - 4s 124ms/step - loss: 1.5967 - accuracy: 0.3027
3/3 [==============================] - 1s 25ms/step - loss: 1.5625 - accuracy: 0.3902
19/19 [==============================] - 4s 102ms/step - loss: 1.5900 - accuracy: 0.3216
3/3 [==============================] - 0s 23ms/step - loss: 1.6086 - accuracy: 0.2195
19/19 [==============================] - 3s 101ms/step - loss: 1.5903 - accuracy: 0.3216
3/3 [==============================] - 0s 19ms/step - loss: 1.5453 - accuracy: 0.3415
19/19 [==============================] - 5s 166ms/step - loss: 1.5851 - accuracy: 0.2324
3/3 [==============================] - 0s 20ms/step - loss: 1.5309 - accuracy: 0.4390
19/19 [==============================] - 4s 105ms/step - loss: 1.5930 - accuracy: 0.2216
3/3 [==============================] - 0s 26ms/step - loss: 1.5463 - accuracy: 0.3171
19/19 [==============================] - 4s 144ms/step - loss: 1.5962 - accuracy: 0.2459
3/3 [==============================] - 0s 20ms/step - loss: 1.5856 - accuracy: 0.2927
19/19 [==============================] - 4s 102ms/step - loss: 1.5966 - accuracy: 0.2243
3/3 [==============================] - 0s 19ms/step - loss: 1.6078 - accuracy: 0.2927
19/19 [==============================] - 4s 103ms/step - loss: 1.5924 - accuracy: 0.3000
3/3 [==============================] - 1s 30ms/step - loss: 1.5622 - accuracy: 0.3659
19/19 [==============================] - 6s 205ms/step - loss: 1.5877 - accuracy: 0.2981
3/3 [==============================] - 0s 44ms/step - loss: 1.5354 - accuracy: 0.3571
19/19 [==============================] - 5s 203ms/step - loss: 1.5778 - accuracy: 0.3135
3/3 [==============================] - 1s 67ms/step - loss: 1.5204 - accuracy: 0.3415
19/19 [==============================] - 6s 203ms/step - loss: 1.5802 - accuracy: 0.3189
3/3 [==============================] - 0s 42ms/step - loss: 1.4713 - accuracy: 0.3902
19/19 [==============================] - 6s 262ms/step - loss: 1.5704 - accuracy: 0.3351
3/3 [==============================] - 0s 43ms/step - loss: 1195171.0000 - accuracy: 0.2195
19/19 [==============================] - 6s 202ms/step - loss: 1517728055269785600.0000 - accuracy: 0.3243
3/3 [==============================] - 0s 43ms/step - loss: 69357369401970524160.0000 - accuracy: 0.3415
19/19 [==============================] - 7s 289ms/step - loss: 1.5872 - accuracy: 0.3027
3/3 [==============================] - 0s 41ms/step - loss: 1.5108 - accuracy: 0.4390
19/19 [==============================] - 6s 204ms/step - loss: 1.5849 - accuracy: 0.2541
3/3 [==============================] - 0s 44ms/step - loss: 1.4917 - accuracy: 0.3171
19/19 [==============================] - 7s 288ms/step - loss: 1.5799 - accuracy: 0.2514
3/3 [==============================] - 0s 39ms/step - loss: 1.5428 - accuracy: 0.2927
19/19 [==============================] - 7s 289ms/step - loss: 1.5785 - accuracy: 0.2595
3/3 [==============================] - 0s 50ms/step - loss: 5838969765888.0000 - accuracy: 0.2927
19/19 [==============================] - 7s 271ms/step - loss: 1.5807 - accuracy: 0.3108
3/3 [==============================] - 1s 42ms/step - loss: 1.5009 - accuracy: 0.3659
19/19 [==============================] - 11s 472ms/step - loss: 1.5862 - accuracy: 0.2629
3/3 [==============================] - 1s 138ms/step - loss: 5962.2021 - accuracy: 0.3571
19/19 [==============================] - 10s 446ms/step - loss: 23903109120.0000 - accuracy: 0.3297
3/3 [==============================] - 1s 114ms/step - loss: 1.5503 - accuracy: 0.3415
19/19 [==============================] - 10s 452ms/step - loss: 978996352.0000 - accuracy: 0.3081
3/3 [==============================] - 1s 95ms/step - loss: 1.5000 - accuracy: 0.3902
19/19 [==============================] - 10s 455ms/step - loss: nan - accuracy: 0.2946
3/3 [==============================] - 1s 75ms/step - loss: nan - accuracy: 0.2683
19/19 [==============================] - 11s 493ms/step - loss: 1295229395140608.0000 - accuracy: 0.3243
3/3 [==============================] - 1s 129ms/step - loss: 1.4857 - accuracy: 0.3415
19/19 [==============================] - 11s 445ms/step - loss: nan - accuracy: 0.3108
3/3 [==============================] - 2s 119ms/step - loss: nan - accuracy: 0.0732
19/19 [==============================] - 11s 456ms/step - loss: nan - accuracy: 0.2595
3/3 [==============================] - 1s 118ms/step - loss: nan - accuracy: 0.0732
19/19 [==============================] - 11s 445ms/step - loss: 7482504118272.0000 - accuracy: 0.3351
3/3 [==============================] - 1s 72ms/step - loss: 1.5681 - accuracy: 0.2927
19/19 [==============================] - 11s 473ms/step - loss: nan - accuracy: 0.3081
3/3 [==============================] - 1s 75ms/step - loss: nan - accuracy: 0.2439
19/19 [==============================] - 11s 522ms/step - loss: 308832192.0000 - accuracy: 0.3108
3/3 [==============================] - 1s 72ms/step - loss: 1.5349 - accuracy: 0.3659
19/19 [==============================] - 3s 76ms/step - loss: 1.5945 - accuracy: 0.3333
3/3 [==============================] - 0s 24ms/step - loss: 1.5720 - accuracy: 0.3571
19/19 [==============================] - 4s 85ms/step - loss: 1.6052 - accuracy: 0.3027
3/3 [==============================] - 0s 16ms/step - loss: 1.5912 - accuracy: 0.3415
19/19 [==============================] - 3s 75ms/step - loss: 1.6057 - accuracy: 0.1351
3/3 [==============================] - 0s 18ms/step - loss: 1.5997 - accuracy: 0.2195
19/19 [==============================] - 3s 77ms/step - loss: 1.5802 - accuracy: 0.3486
3/3 [==============================] - 0s 15ms/step - loss: 1.5982 - accuracy: 0.2195
19/19 [==============================] - 4s 116ms/step - loss: 1.6006 - accuracy: 0.3351
3/3 [==============================] - 0s 18ms/step - loss: 1.5920 - accuracy: 0.3415
19/19 [==============================] - 3s 77ms/step - loss: 1.6028 - accuracy: 0.2189
3/3 [==============================] - 0s 18ms/step - loss: 1.5975 - accuracy: 0.1951
19/19 [==============================] - 3s 76ms/step - loss: 1.6054 - accuracy: 0.3189
3/3 [==============================] - 0s 19ms/step - loss: 1.6011 - accuracy: 0.3171
19/19 [==============================] - 4s 113ms/step - loss: 1.6014 - accuracy: 0.2486
3/3 [==============================] - 0s 15ms/step - loss: 1.5875 - accuracy: 0.3415
19/19 [==============================] - 3s 77ms/step - loss: 1.5995 - accuracy: 0.2973
3/3 [==============================] - 0s 15ms/step - loss: 1.6022 - accuracy: 0.2927
19/19 [==============================] - 3s 78ms/step - loss: 1.6033 - accuracy: 0.2973
3/3 [==============================] - 0s 21ms/step - loss: 1.5892 - accuracy: 0.3659
19/19 [==============================] - 4s 81ms/step - loss: 1.5964 - accuracy: 0.2249
3/3 [==============================] - 0s 16ms/step - loss: 1.5759 - accuracy: 0.2381
19/19 [==============================] - 3s 78ms/step - loss: 1.5915 - accuracy: 0.3135
3/3 [==============================] - 0s 16ms/step - loss: 1.5596 - accuracy: 0.3415
19/19 [==============================] - 3s 82ms/step - loss: 1.6013 - accuracy: 0.2865
3/3 [==============================] - 0s 16ms/step - loss: 1.5781 - accuracy: 0.3902
19/19 [==============================] - 4s 92ms/step - loss: 1.6058 - accuracy: 0.3027
3/3 [==============================] - 0s 17ms/step - loss: 1.6071 - accuracy: 0.2195
19/19 [==============================] - 3s 79ms/step - loss: 1.5884 - accuracy: 0.2595
3/3 [==============================] - 0s 18ms/step - loss: 1.5618 - accuracy: 0.2439
19/19 [==============================] - 3s 80ms/step - loss: 1.6002 - accuracy: 0.3054
3/3 [==============================] - 0s 23ms/step - loss: 1.5790 - accuracy: 0.4390
19/19 [==============================] - 4s 90ms/step - loss: 1.6006 - accuracy: 0.2649
3/3 [==============================] - 0s 17ms/step - loss: 1.5813 - accuracy: 0.3171
19/19 [==============================] - 3s 79ms/step - loss: 1.6038 - accuracy: 0.2351
3/3 [==============================] - 0s 17ms/step - loss: 1.5940 - accuracy: 0.3415
19/19 [==============================] - 3s 80ms/step - loss: 1.5783 - accuracy: 0.2676
3/3 [==============================] - 0s 17ms/step - loss: 1.6211 - accuracy: 0.1707
19/19 [==============================] - 4s 122ms/step - loss: 1.5971 - accuracy: 0.2757
3/3 [==============================] - 0s 18ms/step - loss: 1.5860 - accuracy: 0.3659
19/19 [==============================] - 3s 88ms/step - loss: 1.6056 - accuracy: 0.2114
3/3 [==============================] - 0s 18ms/step - loss: 1.5942 - accuracy: 0.2619
19/19 [==============================] - 3s 87ms/step - loss: 1.5971 - accuracy: 0.3351
3/3 [==============================] - 1s 24ms/step - loss: 1.5759 - accuracy: 0.3415
19/19 [==============================] - 4s 88ms/step - loss: 1.5908 - accuracy: 0.2459
3/3 [==============================] - 0s 18ms/step - loss: 1.5521 - accuracy: 0.3902
19/19 [==============================] - 4s 133ms/step - loss: 1.5847 - accuracy: 0.3486
3/3 [==============================] - 0s 17ms/step - loss: 1.6263 - accuracy: 0.2195
19/19 [==============================] - 4s 108ms/step - loss: 1.5946 - accuracy: 0.3351
3/3 [==============================] - 1s 30ms/step - loss: 1.5722 - accuracy: 0.3415
19/19 [==============================] - 3s 88ms/step - loss: 1.5944 - accuracy: 0.3243
3/3 [==============================] - 0s 19ms/step - loss: 1.5591 - accuracy: 0.4390
19/19 [==============================] - 3s 87ms/step - loss: 1.5980 - accuracy: 0.3189
3/3 [==============================] - 0s 16ms/step - loss: 1.5745 - accuracy: 0.3171
19/19 [==============================] - 4s 127ms/step - loss: 1.5761 - accuracy: 0.2405
3/3 [==============================] - 0s 17ms/step - loss: 1.5703 - accuracy: 0.1463
19/19 [==============================] - 3s 86ms/step - loss: 1.6077 - accuracy: 0.2243
3/3 [==============================] - 0s 17ms/step - loss: 1.6053 - accuracy: 0.2927
19/19 [==============================] - 3s 88ms/step - loss: 1.5919 - accuracy: 0.3027
3/3 [==============================] - 0s 18ms/step - loss: 1.5659 - accuracy: 0.3659
19/19 [==============================] - 5s 134ms/step - loss: 1.5991 - accuracy: 0.2168
3/3 [==============================] - 0s 21ms/step - loss: 1.5808 - accuracy: 0.2619
19/19 [==============================] - 3s 102ms/step - loss: 1.5873 - accuracy: 0.2865
3/3 [==============================] - 1s 19ms/step - loss: 1.5465 - accuracy: 0.3415
19/19 [==============================] - 4s 140ms/step - loss: 1.5965 - accuracy: 0.2595
3/3 [==============================] - 0s 20ms/step - loss: 1.5630 - accuracy: 0.3902
19/19 [==============================] - 4s 103ms/step - loss: 1.5855 - accuracy: 0.3351
3/3 [==============================] - 0s 20ms/step - loss: 1.5993 - accuracy: 0.2195
19/19 [==============================] - 4s 122ms/step - loss: 1.5881 - accuracy: 0.3108
3/3 [==============================] - 1s 32ms/step - loss: 1.5500 - accuracy: 0.3415
19/19 [==============================] - 4s 103ms/step - loss: 1.5980 - accuracy: 0.2919
3/3 [==============================] - 0s 20ms/step - loss: 1.5634 - accuracy: 0.4390
19/19 [==============================] - 4s 102ms/step - loss: 1.5887 - accuracy: 0.3243
3/3 [==============================] - 0s 20ms/step - loss: 1.5468 - accuracy: 0.3171
19/19 [==============================] - 5s 157ms/step - loss: 1.6007 - accuracy: 0.2243
3/3 [==============================] - 0s 20ms/step - loss: 1.5878 - accuracy: 0.3415
19/19 [==============================] - 3s 102ms/step - loss: 1.5894 - accuracy: 0.2595
3/3 [==============================] - 0s 22ms/step - loss: 1.6127 - accuracy: 0.2927
19/19 [==============================] - 4s 123ms/step - loss: 1.5969 - accuracy: 0.2703
3/3 [==============================] - 1s 36ms/step - loss: 1.5612 - accuracy: 0.3659
19/19 [==============================] - 6s 200ms/step - loss: 1.5872 - accuracy: 0.2818
3/3 [==============================] - 0s 46ms/step - loss: 1.5262 - accuracy: 0.3571
19/19 [==============================] - 7s 290ms/step - loss: 1.5601 - accuracy: 0.2946
3/3 [==============================] - 0s 42ms/step - loss: 7889575213006848.0000 - accuracy: 0.3415
19/19 [==============================] - 6s 257ms/step - loss: 1.5807 - accuracy: 0.3189
3/3 [==============================] - 1s 80ms/step - loss: 1.3950 - accuracy: 0.3902
19/19 [==============================] - 6s 207ms/step - loss: 1.5767 - accuracy: 0.3297
3/3 [==============================] - 0s 44ms/step - loss: 1.6163 - accuracy: 0.2195
19/19 [==============================] - 7s 259ms/step - loss: 3658.2605 - accuracy: 0.3351
3/3 [==============================] - 0s 50ms/step - loss: 1.4372 - accuracy: 0.3415
19/19 [==============================] - 6s 204ms/step - loss: 1.5921 - accuracy: 0.2054
3/3 [==============================] - 0s 47ms/step - loss: 1.5324 - accuracy: 0.2683
19/19 [==============================] - 7s 241ms/step - loss: 1.5922 - accuracy: 0.3189
3/3 [==============================] - 0s 44ms/step - loss: 1.5459 - accuracy: 0.3171
19/19 [==============================] - 5s 205ms/step - loss: 1.5811 - accuracy: 0.3216
3/3 [==============================] - 1s 75ms/step - loss: 1.5479 - accuracy: 0.2927
19/19 [==============================] - 6s 203ms/step - loss: 21435042955167531008.0000 - accuracy: 0.2784
3/3 [==============================] - 0s 43ms/step - loss: 882049760637625565184.0000 - accuracy: 0.2927
19/19 [==============================] - 7s 287ms/step - loss: 1.7124 - accuracy: 0.3270
3/3 [==============================] - 0s 44ms/step - loss: 1.4816 - accuracy: 0.3659
19/19 [==============================] - 12s 557ms/step - loss: 1.5653 - accuracy: 0.2737
3/3 [==============================] - 1s 87ms/step - loss: 1.4807 - accuracy: 0.3571
19/19 [==============================] - 12s 531ms/step - loss: nan - accuracy: 0.2595
3/3 [==============================] - 1s 74ms/step - loss: nan - accuracy: 0.0244
19/19 [==============================] - 11s 528ms/step - loss: nan - accuracy: 0.3162
3/3 [==============================] - 1s 80ms/step - loss: nan - accuracy: 0.0488
19/19 [==============================] - 11s 524ms/step - loss: 337593.1562 - accuracy: 0.3486
3/3 [==============================] - 1s 82ms/step - loss: 1.6036 - accuracy: 0.2195
19/19 [==============================] - 11s 514ms/step - loss: nan - accuracy: 0.2541
3/3 [==============================] - 1s 79ms/step - loss: nan - accuracy: 0.0488
19/19 [==============================] - 10s 469ms/step - loss: nan - accuracy: 0.2649
3/3 [==============================] - 2s 127ms/step - loss: nan - accuracy: 0.0732
19/19 [==============================] - 11s 479ms/step - loss: nan - accuracy: 0.2946
3/3 [==============================] - 1s 112ms/step - loss: nan - accuracy: 0.0732
19/19 [==============================] - 10s 446ms/step - loss: 1.5459 - accuracy: 0.2676
3/3 [==============================] - 1s 108ms/step - loss: 1.5362 - accuracy: 0.2927
19/19 [==============================] - 10s 448ms/step - loss: nan - accuracy: 0.2216
3/3 [==============================] - 1s 75ms/step - loss: nan - accuracy: 0.2439
19/19 [==============================] - 10s 463ms/step - loss: nan - accuracy: 0.2811
3/3 [==============================] - 1s 77ms/step - loss: nan - accuracy: 0.0732
8/8 [==============================] - 2s 84ms/step - loss: 1.5995 - accuracy: 0.3333
1/1 [==============================] - 0s 306ms/step - loss: 1.5808 - accuracy: 0.3571
8/8 [==============================] - 2s 83ms/step - loss: 1.5995 - accuracy: 0.3351
1/1 [==============================] - 0s 353ms/step - loss: 1.6049 - accuracy: 0.3415
8/8 [==============================] - 2s 86ms/step - loss: 1.6064 - accuracy: 0.2622
1/1 [==============================] - 0s 320ms/step - loss: 1.5969 - accuracy: 0.3171
8/8 [==============================] - 3s 117ms/step - loss: 1.6023 - accuracy: 0.2459
1/1 [==============================] - 0s 307ms/step - loss: 1.6189 - accuracy: 0.1220
8/8 [==============================] - 2s 84ms/step - loss: 1.6080 - accuracy: 0.1568
1/1 [==============================] - 0s 308ms/step - loss: 1.6063 - accuracy: 0.3415
8/8 [==============================] - 2s 82ms/step - loss: 1.6045 - accuracy: 0.2081
1/1 [==============================] - 0s 301ms/step - loss: 1.5934 - accuracy: 0.2683
8/8 [==============================] - 2s 84ms/step - loss: 1.6002 - accuracy: 0.2216
1/1 [==============================] - 0s 458ms/step - loss: 1.5809 - accuracy: 0.3171
8/8 [==============================] - 4s 83ms/step - loss: 1.6073 - accuracy: 0.3405
1/1 [==============================] - 0s 352ms/step - loss: 1.6062 - accuracy: 0.2927
8/8 [==============================] - 2s 85ms/step - loss: 1.6104 - accuracy: 0.2432
1/1 [==============================] - 0s 327ms/step - loss: 1.6162 - accuracy: 0.1220
8/8 [==============================] - 2s 83ms/step - loss: 1.6088 - accuracy: 0.1973
1/1 [==============================] - 0s 330ms/step - loss: 1.6053 - accuracy: 0.1463
8/8 [==============================] - 3s 128ms/step - loss: 1.6044 - accuracy: 0.2575
1/1 [==============================] - 0s 485ms/step - loss: 1.5955 - accuracy: 0.2619
8/8 [==============================] - 4s 137ms/step - loss: 1.6116 - accuracy: 0.1757
1/1 [==============================] - 0s 444ms/step - loss: 1.6017 - accuracy: 0.3415
8/8 [==============================] - 2s 87ms/step - loss: 1.5998 - accuracy: 0.3297
1/1 [==============================] - 0s 316ms/step - loss: 1.5803 - accuracy: 0.3902
8/8 [==============================] - 2s 90ms/step - loss: 1.5933 - accuracy: 0.3486
1/1 [==============================] - 0s 314ms/step - loss: 1.6102 - accuracy: 0.2195
8/8 [==============================] - 3s 133ms/step - loss: 1.5968 - accuracy: 0.3189
1/1 [==============================] - 1s 505ms/step - loss: 1.5737 - accuracy: 0.3415
8/8 [==============================] - 2s 85ms/step - loss: 1.6081 - accuracy: 0.2649
1/1 [==============================] - 0s 334ms/step - loss: 1.6051 - accuracy: 0.1951
8/8 [==============================] - 2s 88ms/step - loss: 1.5987 - accuracy: 0.2216
1/1 [==============================] - 0s 325ms/step - loss: 1.5792 - accuracy: 0.3171
8/8 [==============================] - 2s 88ms/step - loss: 1.6063 - accuracy: 0.2189
1/1 [==============================] - 0s 305ms/step - loss: 1.6027 - accuracy: 0.1463
8/8 [==============================] - 3s 127ms/step - loss: 1.6056 - accuracy: 0.2838
1/1 [==============================] - 0s 465ms/step - loss: 1.6096 - accuracy: 0.2927
8/8 [==============================] - 4s 88ms/step - loss: 1.6079 - accuracy: 0.2270
1/1 [==============================] - 0s 317ms/step - loss: 1.5956 - accuracy: 0.3659
8/8 [==============================] - 2s 98ms/step - loss: 1.5902 - accuracy: 0.3333
1/1 [==============================] - 0s 312ms/step - loss: 1.5688 - accuracy: 0.3571
8/8 [==============================] - 2s 106ms/step - loss: 1.6025 - accuracy: 0.2459
1/1 [==============================] - 0s 444ms/step - loss: 1.5869 - accuracy: 0.3659
8/8 [==============================] - 3s 97ms/step - loss: 1.6011 - accuracy: 0.3297
1/1 [==============================] - 0s 327ms/step - loss: 1.5815 - accuracy: 0.3902
8/8 [==============================] - 2s 100ms/step - loss: 1.5980 - accuracy: 0.2568
1/1 [==============================] - 0s 328ms/step - loss: 1.6086 - accuracy: 0.2683
8/8 [==============================] - 2s 97ms/step - loss: 1.5992 - accuracy: 0.3351
1/1 [==============================] - 0s 308ms/step - loss: 1.5792 - accuracy: 0.3415
8/8 [==============================] - 3s 142ms/step - loss: 1.6014 - accuracy: 0.3081
1/1 [==============================] - 0s 490ms/step - loss: 1.5872 - accuracy: 0.4390
8/8 [==============================] - 2s 99ms/step - loss: 1.6051 - accuracy: 0.3351
1/1 [==============================] - 0s 309ms/step - loss: 1.5974 - accuracy: 0.3171
8/8 [==============================] - 2s 99ms/step - loss: 1.6090 - accuracy: 0.1784
1/1 [==============================] - 0s 313ms/step - loss: 1.6009 - accuracy: 0.3415
8/8 [==============================] - 2s 100ms/step - loss: 1.6021 - accuracy: 0.2432
1/1 [==============================] - 0s 324ms/step - loss: 1.6110 - accuracy: 0.1220
8/8 [==============================] - 3s 154ms/step - loss: 1.5983 - accuracy: 0.3054
1/1 [==============================] - 0s 500ms/step - loss: 1.5750 - accuracy: 0.3659
8/8 [==============================] - 3s 125ms/step - loss: 1.6033 - accuracy: 0.3333
1/1 [==============================] - 0s 336ms/step - loss: 1.5913 - accuracy: 0.3571
8/8 [==============================] - 4s 130ms/step - loss: 1.5968 - accuracy: 0.2946
1/1 [==============================] - 0s 342ms/step - loss: 1.5829 - accuracy: 0.3415
8/8 [==============================] - 3s 202ms/step - loss: 1.6023 - accuracy: 0.3297
1/1 [==============================] - 0s 464ms/step - loss: 1.5840 - accuracy: 0.3902
8/8 [==============================] - 3s 126ms/step - loss: 1.6024 - accuracy: 0.2811
1/1 [==============================] - 0s 317ms/step - loss: 1.6124 - accuracy: 0.2195
8/8 [==============================] - 3s 130ms/step - loss: 1.6001 - accuracy: 0.2784
1/1 [==============================] - 0s 362ms/step - loss: 1.5822 - accuracy: 0.3415
8/8 [==============================] - 3s 126ms/step - loss: 1.5973 - accuracy: 0.3243
1/1 [==============================] - 0s 319ms/step - loss: 1.5766 - accuracy: 0.4390
8/8 [==============================] - 4s 195ms/step - loss: 1.6063 - accuracy: 0.2865
1/1 [==============================] - 0s 323ms/step - loss: 1.5974 - accuracy: 0.3171
8/8 [==============================] - 3s 128ms/step - loss: 1.5964 - accuracy: 0.3405
1/1 [==============================] - 0s 333ms/step - loss: 1.5901 - accuracy: 0.2927
8/8 [==============================] - 3s 127ms/step - loss: 1.5932 - accuracy: 0.2297
1/1 [==============================] - 0s 338ms/step - loss: 1.6168 - accuracy: 0.1220
8/8 [==============================] - 3s 151ms/step - loss: 1.5939 - accuracy: 0.3324
1/1 [==============================] - 0s 468ms/step - loss: 1.5740 - accuracy: 0.3659
8/8 [==============================] - 5s 304ms/step - loss: 1.6007 - accuracy: 0.2683
1/1 [==============================] - 0s 392ms/step - loss: 1.5827 - accuracy: 0.3571
8/8 [==============================] - 4s 305ms/step - loss: 1.6031 - accuracy: 0.2784
1/1 [==============================] - 0s 385ms/step - loss: 1.5878 - accuracy: 0.3415
8/8 [==============================] - 6s 516ms/step - loss: 1.5972 - accuracy: 0.3297
1/1 [==============================] - 0s 392ms/step - loss: 1.5696 - accuracy: 0.3902
8/8 [==============================] - 5s 302ms/step - loss: 1.5978 - accuracy: 0.2297
1/1 [==============================] - 0s 402ms/step - loss: 1.6112 - accuracy: 0.1220
8/8 [==============================] - 6s 510ms/step - loss: 1.5944 - accuracy: 0.3351
1/1 [==============================] - 0s 408ms/step - loss: 1.5717 - accuracy: 0.3415
8/8 [==============================] - 4s 304ms/step - loss: 1.6007 - accuracy: 0.3243
1/1 [==============================] - 0s 396ms/step - loss: 1.5734 - accuracy: 0.4390
8/8 [==============================] - 4s 349ms/step - loss: 1.6022 - accuracy: 0.2892
1/1 [==============================] - 1s 607ms/step - loss: 1.5841 - accuracy: 0.3171
8/8 [==============================] - 4s 299ms/step - loss: 1.6014 - accuracy: 0.3216
1/1 [==============================] - 0s 402ms/step - loss: 1.5915 - accuracy: 0.2927
8/8 [==============================] - 4s 296ms/step - loss: 1.5976 - accuracy: 0.3405
1/1 [==============================] - 0s 390ms/step - loss: 1.6066 - accuracy: 0.2927
8/8 [==============================] - 6s 480ms/step - loss: 1.5994 - accuracy: 0.3000
1/1 [==============================] - 0s 397ms/step - loss: 1.5785 - accuracy: 0.3659
8/8 [==============================] - 9s 951ms/step - loss: 1.5996 - accuracy: 0.2168
1/1 [==============================] - 1s 705ms/step - loss: 1.5702 - accuracy: 0.2619
8/8 [==============================] - 7s 670ms/step - loss: 1.5885 - accuracy: 0.2946
1/1 [==============================] - 1s 778ms/step - loss: 1.5500 - accuracy: 0.3415
8/8 [==============================] - 7s 653ms/step - loss: 1.5952 - accuracy: 0.3135
1/1 [==============================] - 1s 535ms/step - loss: 1.5497 - accuracy: 0.3902
8/8 [==============================] - 7s 677ms/step - loss: 1.5837 - accuracy: 0.3324
1/1 [==============================] - 0s 497ms/step - loss: 1.6302 - accuracy: 0.2195
8/8 [==============================] - 8s 800ms/step - loss: 1.5957 - accuracy: 0.2973
1/1 [==============================] - 0s 449ms/step - loss: 1.5491 - accuracy: 0.3415
8/8 [==============================] - 10s 881ms/step - loss: 1.5945 - accuracy: 0.2270
1/1 [==============================] - 1s 504ms/step - loss: 1.5498 - accuracy: 0.4390
8/8 [==============================] - 8s 861ms/step - loss: 1.5932 - accuracy: 0.3135
1/1 [==============================] - 1s 554ms/step - loss: 1.5403 - accuracy: 0.3171
8/8 [==============================] - 7s 679ms/step - loss: 1.5960 - accuracy: 0.2486
1/1 [==============================] - 0s 486ms/step - loss: 1.5757 - accuracy: 0.2927
8/8 [==============================] - 8s 775ms/step - loss: 1.5930 - accuracy: 0.2432
1/1 [==============================] - 1s 500ms/step - loss: 1.6170 - accuracy: 0.2927
8/8 [==============================] - 8s 809ms/step - loss: 1.5922 - accuracy: 0.3054
1/1 [==============================] - 0s 482ms/step - loss: 1.5555 - accuracy: 0.3659
8/8 [==============================] - 2s 86ms/step - loss: 1.6058 - accuracy: 0.2818
1/1 [==============================] - 0s 413ms/step - loss: 1.5961 - accuracy: 0.3571
8/8 [==============================] - 3s 93ms/step - loss: 1.6124 - accuracy: 0.1081
1/1 [==============================] - 0s 314ms/step - loss: 1.6064 - accuracy: 0.3415
8/8 [==============================] - 2s 89ms/step - loss: 1.6088 - accuracy: 0.2486
1/1 [==============================] - 0s 316ms/step - loss: 1.6016 - accuracy: 0.3171
8/8 [==============================] - 2s 85ms/step - loss: 1.6031 - accuracy: 0.3486
1/1 [==============================] - 0s 306ms/step - loss: 1.6075 - accuracy: 0.2195
8/8 [==============================] - 2s 83ms/step - loss: 1.6051 - accuracy: 0.3108
1/1 [==============================] - 0s 471ms/step - loss: 1.5993 - accuracy: 0.3415
8/8 [==============================] - 3s 85ms/step - loss: 1.6074 - accuracy: 0.2568
1/1 [==============================] - 0s 313ms/step - loss: 1.6019 - accuracy: 0.4390
8/8 [==============================] - 2s 87ms/step - loss: 1.6044 - accuracy: 0.2568
1/1 [==============================] - 0s 339ms/step - loss: 1.5992 - accuracy: 0.2683
8/8 [==============================] - 4s 89ms/step - loss: 1.6035 - accuracy: 0.2486
1/1 [==============================] - 0s 329ms/step - loss: 1.6042 - accuracy: 0.2927
8/8 [==============================] - 3s 125ms/step - loss: 1.6056 - accuracy: 0.2432
1/1 [==============================] - 0s 319ms/step - loss: 1.6278 - accuracy: 0.1220
8/8 [==============================] - 2s 85ms/step - loss: 1.6080 - accuracy: 0.1757
1/1 [==============================] - 0s 307ms/step - loss: 1.6039 - accuracy: 0.1463
8/8 [==============================] - 2s 89ms/step - loss: 1.6075 - accuracy: 0.3333
1/1 [==============================] - 0s 309ms/step - loss: 1.6027 - accuracy: 0.3571
8/8 [==============================] - 2s 89ms/step - loss: 1.5949 - accuracy: 0.2378
1/1 [==============================] - 0s 394ms/step - loss: 1.5855 - accuracy: 0.1707
8/8 [==============================] - 3s 110ms/step - loss: 1.6068 - accuracy: 0.2757
1/1 [==============================] - 0s 318ms/step - loss: 1.5979 - accuracy: 0.3902
8/8 [==============================] - 2s 90ms/step - loss: 1.6032 - accuracy: 0.2784
1/1 [==============================] - 0s 320ms/step - loss: 1.6105 - accuracy: 0.2195
8/8 [==============================] - 2s 91ms/step - loss: 1.6010 - accuracy: 0.2649
1/1 [==============================] - 0s 316ms/step - loss: 1.5928 - accuracy: 0.2439
8/8 [==============================] - 3s 94ms/step - loss: 1.6100 - accuracy: 0.2270
1/1 [==============================] - 0s 469ms/step - loss: 1.5951 - accuracy: 0.2683
8/8 [==============================] - 3s 88ms/step - loss: 1.6000 - accuracy: 0.3378
1/1 [==============================] - 0s 299ms/step - loss: 1.5950 - accuracy: 0.3171
8/8 [==============================] - 2s 88ms/step - loss: 1.6071 - accuracy: 0.1919
1/1 [==============================] - 0s 303ms/step - loss: 1.5976 - accuracy: 0.3415
8/8 [==============================] - 3s 132ms/step - loss: 1.6028 - accuracy: 0.3324
1/1 [==============================] - 0s 483ms/step - loss: 1.6075 - accuracy: 0.2927
8/8 [==============================] - 5s 115ms/step - loss: 1.6019 - accuracy: 0.2189
1/1 [==============================] - 0s 339ms/step - loss: 1.5890 - accuracy: 0.3415
8/8 [==============================] - 2s 99ms/step - loss: 1.6029 - accuracy: 0.3333
1/1 [==============================] - 0s 316ms/step - loss: 1.5913 - accuracy: 0.3571
8/8 [==============================] - 2s 96ms/step - loss: 1.5958 - accuracy: 0.2378
1/1 [==============================] - 0s 313ms/step - loss: 1.5791 - accuracy: 0.3659
8/8 [==============================] - 3s 148ms/step - loss: 1.6021 - accuracy: 0.3297
1/1 [==============================] - 0s 460ms/step - loss: 1.5868 - accuracy: 0.3902
8/8 [==============================] - 3s 98ms/step - loss: 1.5960 - accuracy: 0.2892
1/1 [==============================] - 0s 335ms/step - loss: 1.5966 - accuracy: 0.2195
8/8 [==============================] - 2s 98ms/step - loss: 1.6025 - accuracy: 0.2865
1/1 [==============================] - 0s 310ms/step - loss: 1.5848 - accuracy: 0.3415
8/8 [==============================] - 2s 98ms/step - loss: 1.6090 - accuracy: 0.1568
1/1 [==============================] - 0s 309ms/step - loss: 1.5978 - accuracy: 0.2683
8/8 [==============================] - 3s 152ms/step - loss: 1.6089 - accuracy: 0.2568
1/1 [==============================] - 0s 388ms/step - loss: 1.5982 - accuracy: 0.2683
8/8 [==============================] - 2s 99ms/step - loss: 1.6035 - accuracy: 0.2405
1/1 [==============================] - 0s 319ms/step - loss: 1.6030 - accuracy: 0.1463
8/8 [==============================] - 2s 95ms/step - loss: 1.6027 - accuracy: 0.3405
1/1 [==============================] - 0s 321ms/step - loss: 1.6056 - accuracy: 0.2927
8/8 [==============================] - 2s 96ms/step - loss: 1.6035 - accuracy: 0.2189
1/1 [==============================] - 0s 371ms/step - loss: 1.5847 - accuracy: 0.3415
8/8 [==============================] - 3s 148ms/step - loss: 1.5984 - accuracy: 0.2981
1/1 [==============================] - 0s 310ms/step - loss: 1.5832 - accuracy: 0.3571
8/8 [==============================] - 4s 131ms/step - loss: 1.6010 - accuracy: 0.2892
1/1 [==============================] - 0s 342ms/step - loss: 1.5961 - accuracy: 0.3415
8/8 [==============================] - 3s 127ms/step - loss: 1.6023 - accuracy: 0.2838
1/1 [==============================] - 0s 335ms/step - loss: 1.5818 - accuracy: 0.3902
8/8 [==============================] - 4s 202ms/step - loss: 1.6012 - accuracy: 0.3486
1/1 [==============================] - 0s 336ms/step - loss: 1.6064 - accuracy: 0.2195
8/8 [==============================] - 3s 129ms/step - loss: 1.6043 - accuracy: 0.3108
1/1 [==============================] - 0s 343ms/step - loss: 1.5922 - accuracy: 0.3415
8/8 [==============================] - 3s 127ms/step - loss: 1.6042 - accuracy: 0.3243
1/1 [==============================] - 0s 322ms/step - loss: 1.5882 - accuracy: 0.4390
8/8 [==============================] - 3s 192ms/step - loss: 1.5975 - accuracy: 0.3378
1/1 [==============================] - 0s 470ms/step - loss: 1.5840 - accuracy: 0.3171
8/8 [==============================] - 3s 127ms/step - loss: 1.5963 - accuracy: 0.2405
1/1 [==============================] - 0s 315ms/step - loss: 1.5943 - accuracy: 0.1463
8/8 [==============================] - 3s 127ms/step - loss: 1.6051 - accuracy: 0.2432
1/1 [==============================] - 0s 326ms/step - loss: 1.6078 - accuracy: 0.2927
8/8 [==============================] - 3s 127ms/step - loss: 1.6006 - accuracy: 0.3000
1/1 [==============================] - 0s 330ms/step - loss: 1.5797 - accuracy: 0.3659
8/8 [==============================] - 5s 396ms/step - loss: 1.5968 - accuracy: 0.3333
1/1 [==============================] - 0s 391ms/step - loss: 1.5763 - accuracy: 0.3571
8/8 [==============================] - 4s 304ms/step - loss: 1.5953 - accuracy: 0.2757
1/1 [==============================] - 0s 376ms/step - loss: 1.5665 - accuracy: 0.3415
8/8 [==============================] - 5s 398ms/step - loss: 1.5996 - accuracy: 0.3297
1/1 [==============================] - 0s 389ms/step - loss: 1.5761 - accuracy: 0.3902
8/8 [==============================] - 5s 304ms/step - loss: 1.5947 - accuracy: 0.3054
1/1 [==============================] - 0s 399ms/step - loss: 1.6068 - accuracy: 0.2195
8/8 [==============================] - 5s 506ms/step - loss: 1.6001 - accuracy: 0.3108
1/1 [==============================] - 1s 561ms/step - loss: 1.5760 - accuracy: 0.3415
8/8 [==============================] - 4s 301ms/step - loss: 1.5980 - accuracy: 0.2270
1/1 [==============================] - 0s 405ms/step - loss: 1.5715 - accuracy: 0.2683
8/8 [==============================] - 4s 300ms/step - loss: 1.6026 - accuracy: 0.2270
1/1 [==============================] - 0s 397ms/step - loss: 1.5809 - accuracy: 0.3171
8/8 [==============================] - 5s 337ms/step - loss: 1.5992 - accuracy: 0.3108
1/1 [==============================] - 0s 409ms/step - loss: 1.5918 - accuracy: 0.2927
8/8 [==============================] - 4s 301ms/step - loss: 1.6018 - accuracy: 0.2703
1/1 [==============================] - 0s 386ms/step - loss: 1.6101 - accuracy: 0.2927
8/8 [==============================] - 5s 382ms/step - loss: 1.6003 - accuracy: 0.2514
1/1 [==============================] - 0s 404ms/step - loss: 1.5825 - accuracy: 0.3659
8/8 [==============================] - 7s 676ms/step - loss: 1.5906 - accuracy: 0.3333
1/1 [==============================] - 1s 504ms/step - loss: 1.5402 - accuracy: 0.3571
8/8 [==============================] - 7s 658ms/step - loss: 1.5913 - accuracy: 0.3351
1/1 [==============================] - 1s 665ms/step - loss: 1.5551 - accuracy: 0.3415
8/8 [==============================] - 8s 708ms/step - loss: 1.5892 - accuracy: 0.3297
1/1 [==============================] - 1s 793ms/step - loss: 1.5172 - accuracy: 0.3902
8/8 [==============================] - 8s 659ms/step - loss: 1.5895 - accuracy: 0.3324
1/1 [==============================] - 1s 753ms/step - loss: 1.6122 - accuracy: 0.2195
8/8 [==============================] - 8s 676ms/step - loss: 1.5983 - accuracy: 0.2973
1/1 [==============================] - 1s 697ms/step - loss: 1.5641 - accuracy: 0.3415
8/8 [==============================] - 8s 665ms/step - loss: 1.5815 - accuracy: 0.3243
1/1 [==============================] - 0s 492ms/step - loss: 1.4892 - accuracy: 0.4390
8/8 [==============================] - 7s 654ms/step - loss: 1.5974 - accuracy: 0.2297
1/1 [==============================] - 0s 489ms/step - loss: 1.5626 - accuracy: 0.3171
8/8 [==============================] - 9s 723ms/step - loss: 1.5933 - accuracy: 0.2865
1/1 [==============================] - 0s 473ms/step - loss: 1.5717 - accuracy: 0.2927
8/8 [==============================] - 8s 807ms/step - loss: 1.5772 - accuracy: 0.3162
1/1 [==============================] - 0s 473ms/step - loss: 1.6654 - accuracy: 0.2927
8/8 [==============================] - 7s 743ms/step - loss: 1.5916 - accuracy: 0.3324
1/1 [==============================] - 1s 755ms/step - loss: 1.5554 - accuracy: 0.3659
5/5 [==============================] - 4s 388ms/step - loss: 1.6063 - accuracy: 0.2579
best parameters for ANN:  {'batch_size': 100, 'nb_epoch': 50, 'neurons': 128}
best score for ANN: 0.34059233367443087

Best parameters for LSTM TFIDF dataset-

best parameters for ANN: {'batch_size': 100, 'nb_epoch': 50, 'neurons': 128}

best score for ANN: 0.34059233367443087

## LSTM Function with tuned parameters

def LSTM_Model_Tuned_TFIDF (X_train, X_test, y_train, y_test):
  in_dim = X_train.shape[1]

  tf.random.set_seed(7)
  embedding_vecor_length = 32

  model = Sequential()
  model.add(Embedding(max_features, embedding_vecor_length, input_length=in_dim))
  model.add(SpatialDropout1D(0.2))
  model.add(LSTM(128))
  model.add(Dense(5 , activation='softmax'))

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=0, patience=3)

  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

  print(model.summary())

  history=model.fit(X_train, y_train_cat, validation_split=0.2, epochs = 50, batch_size = 100, callbacks=[early_stopping])

  # train_score = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  # test_score = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  # result_kfold_df= pd.DataFrame({'model': ['Neural Network'], 'train accuracy': [train_score], 'test accuracy': [test_score] })

  train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  result_kfold_df= pd.DataFrame({'model': ['LSTM'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })


  # print(result_kfold_df)

  hist= pd.DataFrame(history.history)

  for col in hist.columns:
    print(col)
    plt.plot(hist[col])
    plt.plot(hist[col])
    plt.title('model-'+col)
    plt.ylabel(col)
    plt.xlabel('epoch')
    plt.show()

  return result_kfold_df

Tuned LSTM function with TFIDF dataset-

LSTM_Model_Tuned_TFIDF(X_train_tfidf, X_test_tfidf, y_train_tfidf, y_test_tfidf)

Model: "sequential_1505"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1505 (Embedding)  (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_59 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1504 (LSTM)            (None, 128)               82432     
                                                                 
 dense_1504 (Dense)          (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
3/3 [==============================] - 8s 1s/step - loss: 1.6030 - accuracy: 0.2481 - val_loss: 1.5895 - val_accuracy: 0.3333
Epoch 2/50
3/3 [==============================] - 2s 526ms/step - loss: 1.5730 - accuracy: 0.3359 - val_loss: 1.5558 - val_accuracy: 0.3333
Epoch 3/50
3/3 [==============================] - 1s 482ms/step - loss: 1.5158 - accuracy: 0.3359 - val_loss: 1.5618 - val_accuracy: 0.3333
Epoch 4/50
3/3 [==============================] - 1s 478ms/step - loss: 1.4900 - accuracy: 0.3359 - val_loss: 1.5352 - val_accuracy: 0.3333
Epoch 5/50
3/3 [==============================] - 1s 477ms/step - loss: 1.4742 - accuracy: 0.3359 - val_loss: 1.5396 - val_accuracy: 0.3333
Epoch 6/50
3/3 [==============================] - 1s 480ms/step - loss: 1.4603 - accuracy: 0.3359 - val_loss: 1.5830 - val_accuracy: 0.3333
Epoch 7/50
3/3 [==============================] - 2s 548ms/step - loss: 1.4652 - accuracy: 0.3359 - val_loss: 1.5972 - val_accuracy: 0.3333
11/11 [==============================] - 1s 75ms/step
3/3 [==============================] - 0s 60ms/step
11/11 [==============================] - 1s 74ms/step
3/3 [==============================] - 0s 65ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.335366	0.337349	0.168449	0.170194

Running Tuned LSTM function 3 times to check result consistency.

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model_Tuned_TFIDF(X_train_tfidf, X_test_tfidf, y_train_tfidf, y_test_tfidf)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_1506"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1506 (Embedding)  (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_60 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1505 (LSTM)            (None, 128)               82432     
                                                                 
 dense_1505 (Dense)          (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
3/3 [==============================] - 5s 787ms/step - loss: 1.6011 - accuracy: 0.3168 - val_loss: 1.5883 - val_accuracy: 0.3333
Epoch 2/50
3/3 [==============================] - 1s 476ms/step - loss: 1.5722 - accuracy: 0.3359 - val_loss: 1.5594 - val_accuracy: 0.3333
Epoch 3/50
3/3 [==============================] - 1s 477ms/step - loss: 1.5209 - accuracy: 0.3359 - val_loss: 1.5794 - val_accuracy: 0.3333
Epoch 4/50
3/3 [==============================] - 1s 478ms/step - loss: 1.5024 - accuracy: 0.3359 - val_loss: 1.5355 - val_accuracy: 0.3333
Epoch 5/50
3/3 [==============================] - 1s 461ms/step - loss: 1.4675 - accuracy: 0.3359 - val_loss: 1.5378 - val_accuracy: 0.3333
Epoch 6/50
3/3 [==============================] - 1s 472ms/step - loss: 1.4726 - accuracy: 0.3359 - val_loss: 1.5449 - val_accuracy: 0.3333
Epoch 7/50
3/3 [==============================] - 2s 791ms/step - loss: 1.4670 - accuracy: 0.3359 - val_loss: 1.5629 - val_accuracy: 0.3333
11/11 [==============================] - 2s 140ms/step
3/3 [==============================] - 0s 63ms/step
11/11 [==============================] - 1s 75ms/step
3/3 [==============================] - 0s 63ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_1507"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1507 (Embedding)  (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_61 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1506 (LSTM)            (None, 128)               82432     
                                                                 
 dense_1506 (Dense)          (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
3/3 [==============================] - 4s 780ms/step - loss: 1.6026 - accuracy: 0.2481 - val_loss: 1.5882 - val_accuracy: 0.3333
Epoch 2/50
3/3 [==============================] - 1s 475ms/step - loss: 1.5678 - accuracy: 0.3206 - val_loss: 1.5531 - val_accuracy: 0.3333
Epoch 3/50
3/3 [==============================] - 2s 821ms/step - loss: 1.5051 - accuracy: 0.3359 - val_loss: 1.6015 - val_accuracy: 0.3333
Epoch 4/50
3/3 [==============================] - 2s 696ms/step - loss: 1.5011 - accuracy: 0.3359 - val_loss: 1.5500 - val_accuracy: 0.3333
Epoch 5/50
3/3 [==============================] - 1s 478ms/step - loss: 1.4630 - accuracy: 0.3359 - val_loss: 1.5409 - val_accuracy: 0.3333
Epoch 6/50
3/3 [==============================] - 1s 490ms/step - loss: 1.4718 - accuracy: 0.3359 - val_loss: 1.5457 - val_accuracy: 0.3333
Epoch 7/50
3/3 [==============================] - 1s 471ms/step - loss: 1.4647 - accuracy: 0.3359 - val_loss: 1.5592 - val_accuracy: 0.3333
Epoch 8/50
3/3 [==============================] - 1s 485ms/step - loss: 1.4639 - accuracy: 0.3359 - val_loss: 1.5859 - val_accuracy: 0.3333
11/11 [==============================] - 2s 133ms/step
3/3 [==============================] - 1s 145ms/step
11/11 [==============================] - 2s 200ms/step
3/3 [==============================] - 0s 119ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_1508"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1508 (Embedding)  (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_62 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1507 (LSTM)            (None, 128)               82432     
                                                                 
 dense_1507 (Dense)          (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
3/3 [==============================] - 8s 1s/step - loss: 1.6038 - accuracy: 0.2099 - val_loss: 1.5910 - val_accuracy: 0.3333
Epoch 2/50
3/3 [==============================] - 2s 753ms/step - loss: 1.5751 - accuracy: 0.3244 - val_loss: 1.5706 - val_accuracy: 0.3333
Epoch 3/50
3/3 [==============================] - 3s 950ms/step - loss: 1.5351 - accuracy: 0.3359 - val_loss: 1.5448 - val_accuracy: 0.3333
Epoch 4/50
3/3 [==============================] - 3s 1s/step - loss: 1.4991 - accuracy: 0.3359 - val_loss: 1.5639 - val_accuracy: 0.3333
Epoch 5/50
3/3 [==============================] - 2s 731ms/step - loss: 1.4649 - accuracy: 0.3359 - val_loss: 1.5516 - val_accuracy: 0.3333
Epoch 6/50
3/3 [==============================] - 2s 801ms/step - loss: 1.4577 - accuracy: 0.3359 - val_loss: 1.5578 - val_accuracy: 0.3333
11/11 [==============================] - 2s 100ms/step
3/3 [==============================] - 1s 158ms/step
11/11 [==============================] - 3s 221ms/step
3/3 [==============================] - 0s 125ms/step
loss

accuracy

val_loss

val_accuracy

Result of all runs:   model  train accuracy  test accuracy  train F1 score  test F1 score
0  LSTM        0.335366       0.337349        0.168449       0.170194
1  LSTM        0.335366       0.337349        0.168449       0.170194
2  LSTM        0.335366       0.337349        0.168449       0.170194

Result is consistent for 3 runs.

Tuned LSTM function with TFIDF Smote dataset-

LSTM_Model_Tuned_TFIDF(X_train_tfidf_smote, X_test_tfidf, y_train_tfidf_smote, y_test_tfidf)

Model: "sequential_1509"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1509 (Embedding)  (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_63 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1508 (LSTM)            (None, 128)               82432     
                                                                 
 dense_1508 (Dense)          (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
5/5 [==============================] - 14s 1s/step - loss: 1.5993 - accuracy: 0.2455 - val_loss: 1.7788 - val_accuracy: 0.0000e+00
Epoch 2/50
5/5 [==============================] - 4s 820ms/step - loss: 1.5621 - accuracy: 0.2159 - val_loss: 2.6629 - val_accuracy: 0.0000e+00
Epoch 3/50
5/5 [==============================] - 6s 1s/step - loss: 1.5315 - accuracy: 0.2568 - val_loss: 2.4662 - val_accuracy: 0.0000e+00
Epoch 4/50
5/5 [==============================] - 5s 910ms/step - loss: 1.5235 - accuracy: 0.2500 - val_loss: 2.3305 - val_accuracy: 0.0000e+00
18/18 [==============================] - 5s 173ms/step
3/3 [==============================] - 1s 149ms/step
18/18 [==============================] - 3s 133ms/step
3/3 [==============================] - 0s 126ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.2	0.228916	0.066667	0.085282

Running Tuned LSTM function 3 times to check result consistency.

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model_Tuned_TFIDF(X_train_tfidf_smote, X_test_tfidf, y_train_tfidf_smote, y_test_tfidf)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_1510"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1510 (Embedding)  (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_64 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1509 (LSTM)            (None, 128)               82432     
                                                                 
 dense_1509 (Dense)          (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
5/5 [==============================] - 10s 1s/step - loss: 1.5986 - accuracy: 0.2682 - val_loss: 1.7743 - val_accuracy: 0.0000e+00
Epoch 2/50
5/5 [==============================] - 5s 1s/step - loss: 1.5604 - accuracy: 0.2545 - val_loss: 2.7851 - val_accuracy: 0.0000e+00
Epoch 3/50
5/5 [==============================] - 5s 1s/step - loss: 1.5348 - accuracy: 0.2455 - val_loss: 2.3475 - val_accuracy: 0.0000e+00
Epoch 4/50
5/5 [==============================] - 4s 859ms/step - loss: 1.5257 - accuracy: 0.2364 - val_loss: 2.2941 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 77ms/step
3/3 [==============================] - 0s 70ms/step
18/18 [==============================] - 1s 78ms/step
3/3 [==============================] - 0s 64ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_1511"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1511 (Embedding)  (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_65 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1510 (LSTM)            (None, 128)               82432     
                                                                 
 dense_1510 (Dense)          (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
5/5 [==============================] - 6s 605ms/step - loss: 1.6046 - accuracy: 0.2045 - val_loss: 1.7158 - val_accuracy: 0.0000e+00
Epoch 2/50
5/5 [==============================] - 2s 459ms/step - loss: 1.5756 - accuracy: 0.2500 - val_loss: 2.0580 - val_accuracy: 0.0000e+00
Epoch 3/50
5/5 [==============================] - 2s 461ms/step - loss: 1.5283 - accuracy: 0.2500 - val_loss: 2.8867 - val_accuracy: 0.0000e+00
Epoch 4/50
5/5 [==============================] - 2s 464ms/step - loss: 1.5230 - accuracy: 0.2500 - val_loss: 2.2598 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 144ms/step
3/3 [==============================] - 0s 116ms/step
18/18 [==============================] - 1s 78ms/step
3/3 [==============================] - 0s 66ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_1512"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1512 (Embedding)  (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_66 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1511 (LSTM)            (None, 128)               82432     
                                                                 
 dense_1511 (Dense)          (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
5/5 [==============================] - 5s 601ms/step - loss: 1.5964 - accuracy: 0.2182 - val_loss: 1.8071 - val_accuracy: 0.0000e+00
Epoch 2/50
5/5 [==============================] - 4s 808ms/step - loss: 1.5553 - accuracy: 0.2455 - val_loss: 2.8185 - val_accuracy: 0.0000e+00
Epoch 3/50
5/5 [==============================] - 2s 470ms/step - loss: 1.5346 - accuracy: 0.2614 - val_loss: 2.4064 - val_accuracy: 0.0000e+00
Epoch 4/50
5/5 [==============================] - 2s 472ms/step - loss: 1.5246 - accuracy: 0.2477 - val_loss: 2.2854 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 81ms/step
3/3 [==============================] - 0s 73ms/step
18/18 [==============================] - 2s 83ms/step
3/3 [==============================] - 0s 67ms/step
loss

accuracy

val_loss

val_accuracy

Result of all runs:   model  train accuracy  test accuracy  train F1 score  test F1 score
0  LSTM             0.2       0.228916        0.066667       0.085282
1  LSTM             0.2       0.108434        0.066667       0.021215
2  LSTM             0.2       0.228916        0.066667       0.085282

F1 score is dipping for a run. It means the result is a little bit inconsistent. It needs more repeatation to under the change.

Tuned LSTM function with TFIDF Full dataset-

LSTM_Model_Tuned_TFIDF(X_train_tfidffull, X_test_tfidffull, y_train_tfidffull, y_test_tfidffull)

Model: "sequential_1513"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1513 (Embedding)  (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_67 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1512 (LSTM)            (None, 128)               82432     
                                                                 
 dense_1512 (Dense)          (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
3/3 [==============================] - 6s 924ms/step - loss: 1.5962 - accuracy: 0.3168 - val_loss: 1.5802 - val_accuracy: 0.3333
Epoch 2/50
3/3 [==============================] - 2s 903ms/step - loss: 1.5561 - accuracy: 0.3359 - val_loss: 1.5410 - val_accuracy: 0.3333
Epoch 3/50
3/3 [==============================] - 3s 1s/step - loss: 1.4955 - accuracy: 0.3359 - val_loss: 1.5628 - val_accuracy: 0.3333
Epoch 4/50
3/3 [==============================] - 2s 541ms/step - loss: 1.4749 - accuracy: 0.3359 - val_loss: 1.5408 - val_accuracy: 0.3333
Epoch 5/50
3/3 [==============================] - 2s 674ms/step - loss: 1.4638 - accuracy: 0.3359 - val_loss: 1.5508 - val_accuracy: 0.3333
Epoch 6/50
3/3 [==============================] - 3s 943ms/step - loss: 1.4612 - accuracy: 0.3015 - val_loss: 1.5719 - val_accuracy: 0.2121
Epoch 7/50
3/3 [==============================] - 2s 536ms/step - loss: 1.4620 - accuracy: 0.3015 - val_loss: 1.5880 - val_accuracy: 0.3333
11/11 [==============================] - 1s 84ms/step
3/3 [==============================] - 0s 78ms/step
11/11 [==============================] - 1s 83ms/step
3/3 [==============================] - 0s 71ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.335366	0.337349	0.168449	0.170194

3 times run-

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model_Tuned_TFIDF(X_train_tfidffull, X_test_tfidffull, y_train_tfidffull, y_test_tfidffull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_1514"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1514 (Embedding)  (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_68 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1513 (LSTM)            (None, 128)               82432     
                                                                 
 dense_1513 (Dense)          (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
3/3 [==============================] - 5s 833ms/step - loss: 1.6020 - accuracy: 0.2939 - val_loss: 1.5868 - val_accuracy: 0.3333
Epoch 2/50
3/3 [==============================] - 2s 531ms/step - loss: 1.5668 - accuracy: 0.3359 - val_loss: 1.5581 - val_accuracy: 0.3333
Epoch 3/50
3/3 [==============================] - 2s 528ms/step - loss: 1.5101 - accuracy: 0.3359 - val_loss: 1.5696 - val_accuracy: 0.3333
Epoch 4/50
3/3 [==============================] - 2s 515ms/step - loss: 1.5915 - accuracy: 0.3359 - val_loss: 1.5720 - val_accuracy: 0.3333
Epoch 5/50
3/3 [==============================] - 2s 532ms/step - loss: 1.4662 - accuracy: 0.3359 - val_loss: 1.5405 - val_accuracy: 0.3333
Epoch 6/50
3/3 [==============================] - 2s 761ms/step - loss: 1.4841 - accuracy: 0.3359 - val_loss: 1.5451 - val_accuracy: 0.3333
Epoch 7/50
3/3 [==============================] - 3s 884ms/step - loss: 1.4921 - accuracy: 0.3321 - val_loss: 1.5469 - val_accuracy: 0.3333
Epoch 8/50
3/3 [==============================] - 2s 540ms/step - loss: 1.4853 - accuracy: 0.2977 - val_loss: 1.5484 - val_accuracy: 0.2121
11/11 [==============================] - 1s 83ms/step
3/3 [==============================] - 0s 71ms/step
11/11 [==============================] - 1s 119ms/step
3/3 [==============================] - 0s 115ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_1515"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1515 (Embedding)  (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_69 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1514 (LSTM)            (None, 128)               82432     
                                                                 
 dense_1514 (Dense)          (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
3/3 [==============================] - 5s 837ms/step - loss: 1.6060 - accuracy: 0.2252 - val_loss: 1.5902 - val_accuracy: 0.3333
Epoch 2/50
3/3 [==============================] - 2s 526ms/step - loss: 1.5714 - accuracy: 0.3359 - val_loss: 1.5611 - val_accuracy: 0.3333
Epoch 3/50
3/3 [==============================] - 2s 549ms/step - loss: 1.5183 - accuracy: 0.3359 - val_loss: 1.5700 - val_accuracy: 0.3333
Epoch 4/50
3/3 [==============================] - 3s 968ms/step - loss: 1.5669 - accuracy: 0.3359 - val_loss: 1.5845 - val_accuracy: 0.3333
Epoch 5/50
3/3 [==============================] - 2s 602ms/step - loss: 1.4661 - accuracy: 0.3359 - val_loss: 1.5397 - val_accuracy: 0.3333
Epoch 6/50
3/3 [==============================] - 2s 558ms/step - loss: 1.4779 - accuracy: 0.3359 - val_loss: 1.5426 - val_accuracy: 0.3333
Epoch 7/50
3/3 [==============================] - 2s 542ms/step - loss: 1.4828 - accuracy: 0.3359 - val_loss: 1.5444 - val_accuracy: 0.3333
Epoch 8/50
3/3 [==============================] - 2s 533ms/step - loss: 1.4756 - accuracy: 0.3359 - val_loss: 1.5498 - val_accuracy: 0.3333
11/11 [==============================] - 1s 82ms/step
3/3 [==============================] - 0s 71ms/step
11/11 [==============================] - 1s 85ms/step
3/3 [==============================] - 0s 73ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_1516"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1516 (Embedding)  (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_70 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1515 (LSTM)            (None, 128)               82432     
                                                                 
 dense_1515 (Dense)          (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
3/3 [==============================] - 5s 881ms/step - loss: 1.6071 - accuracy: 0.2176 - val_loss: 1.5909 - val_accuracy: 0.3333
Epoch 2/50
3/3 [==============================] - 2s 542ms/step - loss: 1.5767 - accuracy: 0.3359 - val_loss: 1.5661 - val_accuracy: 0.3333
Epoch 3/50
3/3 [==============================] - 2s 554ms/step - loss: 1.5309 - accuracy: 0.3359 - val_loss: 1.5369 - val_accuracy: 0.3333
Epoch 4/50
3/3 [==============================] - 2s 544ms/step - loss: 1.5149 - accuracy: 0.3359 - val_loss: 1.5482 - val_accuracy: 0.3333
Epoch 5/50
3/3 [==============================] - 5s 2s/step - loss: 1.4677 - accuracy: 0.3359 - val_loss: 1.5376 - val_accuracy: 0.3333
Epoch 6/50
3/3 [==============================] - 2s 539ms/step - loss: 1.4702 - accuracy: 0.3359 - val_loss: 1.5415 - val_accuracy: 0.3333
11/11 [==============================] - 1s 84ms/step
3/3 [==============================] - 0s 65ms/step
11/11 [==============================] - 1s 90ms/step
3/3 [==============================] - 0s 76ms/step
loss

accuracy

val_loss

val_accuracy

Result of all runs:   model  train accuracy  test accuracy  train F1 score  test F1 score
0  LSTM        0.335366       0.337349        0.168449       0.170194
1  LSTM        0.335366       0.337349        0.168449       0.170194
2  LSTM        0.259146       0.253012        0.106670       0.102178

Result-

There is definitely some changes in F1 score. Loss is modifying as well. Accuracy is remaining constant.

With TFIDF smote dataset-

LSTM_Model_Tuned_TFIDF(X_train_tfidffull_smote, X_test_tfidffull, y_train_tfidffull_smote, y_test_tfidffull)

Model: "sequential_1517"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1517 (Embedding)  (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_71 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1516 (LSTM)            (None, 128)               82432     
                                                                 
 dense_1516 (Dense)          (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
5/5 [==============================] - 7s 1s/step - loss: 1.5974 - accuracy: 0.2091 - val_loss: 1.7919 - val_accuracy: 0.0000e+00
Epoch 2/50
5/5 [==============================] - 3s 532ms/step - loss: 1.5567 - accuracy: 0.2364 - val_loss: 3.0235 - val_accuracy: 0.0000e+00
Epoch 3/50
5/5 [==============================] - 3s 538ms/step - loss: 1.5393 - accuracy: 0.2477 - val_loss: 2.4028 - val_accuracy: 0.0000e+00
Epoch 4/50
5/5 [==============================] - 3s 521ms/step - loss: 1.5256 - accuracy: 0.2432 - val_loss: 2.2633 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 86ms/step
3/3 [==============================] - 0s 71ms/step
18/18 [==============================] - 2s 86ms/step
3/3 [==============================] - 0s 73ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.2	0.108434	0.066667	0.021215

3 times run-

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model_Tuned_TFIDF(X_train_tfidffull_smote, X_test_tfidffull, y_train_tfidffull_smote, y_test_tfidffull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_1518"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1518 (Embedding)  (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_72 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1517 (LSTM)            (None, 128)               82432     
                                                                 
 dense_1517 (Dense)          (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
5/5 [==============================] - 6s 687ms/step - loss: 1.5968 - accuracy: 0.2545 - val_loss: 1.7739 - val_accuracy: 0.0000e+00
Epoch 2/50
5/5 [==============================] - 3s 526ms/step - loss: 1.5599 - accuracy: 0.2500 - val_loss: 2.9363 - val_accuracy: 0.0000e+00
Epoch 3/50
5/5 [==============================] - 3s 558ms/step - loss: 1.5407 - accuracy: 0.2386 - val_loss: 2.4459 - val_accuracy: 0.0000e+00
Epoch 4/50
5/5 [==============================] - 4s 785ms/step - loss: 1.5243 - accuracy: 0.2682 - val_loss: 2.2772 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 88ms/step
3/3 [==============================] - 0s 71ms/step
18/18 [==============================] - 2s 86ms/step
3/3 [==============================] - 0s 72ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_1519"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1519 (Embedding)  (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_73 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1518 (LSTM)            (None, 128)               82432     
                                                                 
 dense_1518 (Dense)          (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
5/5 [==============================] - 8s 1s/step - loss: 1.6032 - accuracy: 0.2000 - val_loss: 1.7457 - val_accuracy: 0.0000e+00
Epoch 2/50
5/5 [==============================] - 3s 523ms/step - loss: 1.5683 - accuracy: 0.2477 - val_loss: 2.5163 - val_accuracy: 0.0000e+00
Epoch 3/50
5/5 [==============================] - 3s 529ms/step - loss: 1.5319 - accuracy: 0.2500 - val_loss: 2.3407 - val_accuracy: 0.0000e+00
Epoch 4/50
5/5 [==============================] - 3s 523ms/step - loss: 1.5266 - accuracy: 0.2545 - val_loss: 2.3502 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 120ms/step
3/3 [==============================] - 0s 71ms/step
18/18 [==============================] - 2s 89ms/step
3/3 [==============================] - 0s 73ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_1520"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1520 (Embedding)  (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_74 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1519 (LSTM)            (None, 128)               82432     
                                                                 
 dense_1519 (Dense)          (None, 5)                 645       
                                                                 
=================================================================
Total params: 89,477
Trainable params: 89,477
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/50
5/5 [==============================] - 6s 677ms/step - loss: 1.6018 - accuracy: 0.2068 - val_loss: 1.7349 - val_accuracy: 0.0000e+00
Epoch 2/50
5/5 [==============================] - 3s 513ms/step - loss: 1.5717 - accuracy: 0.2273 - val_loss: 2.2672 - val_accuracy: 0.0000e+00
Epoch 3/50
5/5 [==============================] - 3s 541ms/step - loss: 1.5249 - accuracy: 0.2568 - val_loss: 2.5961 - val_accuracy: 0.0000e+00
Epoch 4/50
5/5 [==============================] - 4s 942ms/step - loss: 1.5192 - accuracy: 0.2500 - val_loss: 2.5126 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 88ms/step
3/3 [==============================] - 0s 70ms/step
18/18 [==============================] - 3s 159ms/step
3/3 [==============================] - 0s 115ms/step
loss

accuracy

val_loss

val_accuracy

Result of all runs:   model  train accuracy  test accuracy  train F1 score  test F1 score
0  LSTM             0.2       0.228916        0.066667       0.085282
1  LSTM             0.2       0.228916        0.066667       0.085282
2  LSTM             0.2       0.108434        0.066667       0.021215

Observations-

1. Accuracy is remaining around 33% and F1 score around 17%.
2. Similar results like Countvectorizer.

Word2vec Dataset-

Tuned_LSTM(X_wv_df, y_wv_df)

4/4 [==============================] - 2s 95ms/step - loss: 1.6057 - accuracy: 0.2304
1/1 [==============================] - 0s 320ms/step - loss: 1.6033 - accuracy: 0.2381
4/4 [==============================] - 2s 94ms/step - loss: 1.6099 - accuracy: 0.2378
1/1 [==============================] - 0s 346ms/step - loss: 1.6063 - accuracy: 0.3659
4/4 [==============================] - 2s 89ms/step - loss: 1.6088 - accuracy: 0.2162
1/1 [==============================] - 0s 446ms/step - loss: 1.6041 - accuracy: 0.3902
4/4 [==============================] - 3s 97ms/step - loss: 1.6064 - accuracy: 0.2432
1/1 [==============================] - 0s 319ms/step - loss: 1.6090 - accuracy: 0.1220
4/4 [==============================] - 2s 92ms/step - loss: 1.5998 - accuracy: 0.2378
1/1 [==============================] - 0s 315ms/step - loss: 1.5846 - accuracy: 0.3415
4/4 [==============================] - 2s 95ms/step - loss: 1.6088 - accuracy: 0.2270
1/1 [==============================] - 0s 336ms/step - loss: 1.6069 - accuracy: 0.2683
4/4 [==============================] - 2s 140ms/step - loss: 1.6139 - accuracy: 0.2216
1/1 [==============================] - 0s 475ms/step - loss: 1.6070 - accuracy: 0.3171
4/4 [==============================] - 3s 96ms/step - loss: 1.6155 - accuracy: 0.0730
1/1 [==============================] - 0s 313ms/step - loss: 1.6117 - accuracy: 0.1220
4/4 [==============================] - 2s 89ms/step - loss: 1.6148 - accuracy: 0.0595
1/1 [==============================] - 0s 323ms/step - loss: 1.6077 - accuracy: 0.1707
4/4 [==============================] - 2s 91ms/step - loss: 1.6088 - accuracy: 0.3324
1/1 [==============================] - 0s 335ms/step - loss: 1.6017 - accuracy: 0.3659
4/4 [==============================] - 2s 92ms/step - loss: 1.6077 - accuracy: 0.2683
1/1 [==============================] - 0s 324ms/step - loss: 1.6050 - accuracy: 0.3571
4/4 [==============================] - 3s 154ms/step - loss: 1.6084 - accuracy: 0.2703
1/1 [==============================] - 0s 468ms/step - loss: 1.6044 - accuracy: 0.3415
4/4 [==============================] - 2s 102ms/step - loss: 1.6059 - accuracy: 0.2757
1/1 [==============================] - 0s 310ms/step - loss: 1.5954 - accuracy: 0.3902
4/4 [==============================] - 2s 102ms/step - loss: 1.6061 - accuracy: 0.2568
1/1 [==============================] - 0s 344ms/step - loss: 1.6085 - accuracy: 0.2683
4/4 [==============================] - 2s 95ms/step - loss: 1.6027 - accuracy: 0.2946
1/1 [==============================] - 0s 334ms/step - loss: 1.5978 - accuracy: 0.2439
4/4 [==============================] - 2s 138ms/step - loss: 1.6049 - accuracy: 0.3081
1/1 [==============================] - 0s 430ms/step - loss: 1.5974 - accuracy: 0.4390
4/4 [==============================] - 2s 98ms/step - loss: 1.6054 - accuracy: 0.1730
1/1 [==============================] - 1s 1s/step - loss: 1.5947 - accuracy: 0.3171
4/4 [==============================] - 2s 105ms/step - loss: 1.6056 - accuracy: 0.2919
1/1 [==============================] - 0s 320ms/step - loss: 1.6061 - accuracy: 0.2927
4/4 [==============================] - 2s 94ms/step - loss: 1.5975 - accuracy: 0.3405
1/1 [==============================] - 0s 320ms/step - loss: 1.6012 - accuracy: 0.2927
4/4 [==============================] - 3s 145ms/step - loss: 1.6098 - accuracy: 0.2189
1/1 [==============================] - 1s 521ms/step - loss: 1.6032 - accuracy: 0.3415
4/4 [==============================] - 2s 113ms/step - loss: 1.6045 - accuracy: 0.2575
1/1 [==============================] - 0s 346ms/step - loss: 1.5994 - accuracy: 0.2619
4/4 [==============================] - 2s 112ms/step - loss: 1.6037 - accuracy: 0.2919
1/1 [==============================] - 0s 320ms/step - loss: 1.5963 - accuracy: 0.3415
4/4 [==============================] - 2s 118ms/step - loss: 1.6085 - accuracy: 0.2162
1/1 [==============================] - 0s 326ms/step - loss: 1.5979 - accuracy: 0.3902
4/4 [==============================] - 2s 171ms/step - loss: 1.6102 - accuracy: 0.2432
1/1 [==============================] - 0s 462ms/step - loss: 1.6150 - accuracy: 0.1220
4/4 [==============================] - 2s 113ms/step - loss: 1.6062 - accuracy: 0.2162
1/1 [==============================] - 0s 312ms/step - loss: 1.5939 - accuracy: 0.3659
4/4 [==============================] - 2s 113ms/step - loss: 1.6086 - accuracy: 0.2946
1/1 [==============================] - 0s 319ms/step - loss: 1.6028 - accuracy: 0.4390
4/4 [==============================] - 2s 113ms/step - loss: 1.6009 - accuracy: 0.3378
1/1 [==============================] - 0s 326ms/step - loss: 1.5932 - accuracy: 0.3171
4/4 [==============================] - 2s 116ms/step - loss: 1.6024 - accuracy: 0.2486
1/1 [==============================] - 0s 470ms/step - loss: 1.6016 - accuracy: 0.2927
4/4 [==============================] - 3s 117ms/step - loss: 1.6145 - accuracy: 0.1541
1/1 [==============================] - 0s 323ms/step - loss: 1.6105 - accuracy: 0.1707
4/4 [==============================] - 2s 114ms/step - loss: 1.6061 - accuracy: 0.2595
1/1 [==============================] - 0s 334ms/step - loss: 1.6010 - accuracy: 0.3659
4/4 [==============================] - 2s 206ms/step - loss: 1.6034 - accuracy: 0.2575
1/1 [==============================] - 0s 352ms/step - loss: 1.5924 - accuracy: 0.2619
4/4 [==============================] - 3s 376ms/step - loss: 1.6032 - accuracy: 0.2541
1/1 [==============================] - 0s 373ms/step - loss: 1.6009 - accuracy: 0.3415
4/4 [==============================] - 3s 214ms/step - loss: 1.6097 - accuracy: 0.2486
1/1 [==============================] - 0s 345ms/step - loss: 1.5985 - accuracy: 0.3902
4/4 [==============================] - 2s 211ms/step - loss: 1.6045 - accuracy: 0.3054
1/1 [==============================] - 0s 322ms/step - loss: 1.6101 - accuracy: 0.2195
4/4 [==============================] - 2s 208ms/step - loss: 1.6071 - accuracy: 0.2378
1/1 [==============================] - 0s 489ms/step - loss: 1.5989 - accuracy: 0.3415
4/4 [==============================] - 3s 203ms/step - loss: 1.6079 - accuracy: 0.2270
1/1 [==============================] - 0s 332ms/step - loss: 1.5974 - accuracy: 0.2683
4/4 [==============================] - 2s 220ms/step - loss: 1.5992 - accuracy: 0.3378
1/1 [==============================] - 0s 332ms/step - loss: 1.5876 - accuracy: 0.3171
4/4 [==============================] - 3s 217ms/step - loss: 1.6047 - accuracy: 0.1459
1/1 [==============================] - 0s 336ms/step - loss: 1.6015 - accuracy: 0.1463
4/4 [==============================] - 3s 392ms/step - loss: 1.6059 - accuracy: 0.2676
1/1 [==============================] - 1s 534ms/step - loss: 1.6118 - accuracy: 0.1707
4/4 [==============================] - 3s 213ms/step - loss: 1.6055 - accuracy: 0.2703
1/1 [==============================] - 0s 317ms/step - loss: 1.6019 - accuracy: 0.1463
4/4 [==============================] - 6s 530ms/step - loss: 1.6036 - accuracy: 0.3089
1/1 [==============================] - 0s 430ms/step - loss: 1.5941 - accuracy: 0.3571
4/4 [==============================] - 5s 514ms/step - loss: 1.6050 - accuracy: 0.2703
1/1 [==============================] - 0s 398ms/step - loss: 1.5963 - accuracy: 0.3415
4/4 [==============================] - 3s 438ms/step - loss: 1.6066 - accuracy: 0.2000
1/1 [==============================] - 0s 403ms/step - loss: 1.5927 - accuracy: 0.2195
4/4 [==============================] - 4s 540ms/step - loss: 1.6007 - accuracy: 0.3054
1/1 [==============================] - 1s 614ms/step - loss: 1.6049 - accuracy: 0.2195
4/4 [==============================] - 4s 441ms/step - loss: 1.6061 - accuracy: 0.2757
1/1 [==============================] - 0s 416ms/step - loss: 1.5941 - accuracy: 0.3415
4/4 [==============================] - 4s 449ms/step - loss: 1.6068 - accuracy: 0.2514
1/1 [==============================] - 0s 407ms/step - loss: 1.5968 - accuracy: 0.1951
4/4 [==============================] - 5s 605ms/step - loss: 1.6050 - accuracy: 0.2216
1/1 [==============================] - 0s 420ms/step - loss: 1.5935 - accuracy: 0.3171
4/4 [==============================] - 3s 437ms/step - loss: 1.6034 - accuracy: 0.2459
1/1 [==============================] - 0s 405ms/step - loss: 1.6006 - accuracy: 0.2927
4/4 [==============================] - 5s 710ms/step - loss: 1.6066 - accuracy: 0.2838
1/1 [==============================] - 0s 402ms/step - loss: 1.6077 - accuracy: 0.2927
4/4 [==============================] - 3s 418ms/step - loss: 1.6024 - accuracy: 0.2703
1/1 [==============================] - 0s 428ms/step - loss: 1.5975 - accuracy: 0.1463
4/4 [==============================] - 7s 2s/step - loss: 1.6007 - accuracy: 0.3089
1/1 [==============================] - 0s 481ms/step - loss: 1.5832 - accuracy: 0.3571
4/4 [==============================] - 6s 1s/step - loss: 1.6004 - accuracy: 0.3108
1/1 [==============================] - 1s 781ms/step - loss: 1.5901 - accuracy: 0.3415
4/4 [==============================] - 7s 999ms/step - loss: 1.5997 - accuracy: 0.3297
1/1 [==============================] - 1s 525ms/step - loss: 1.5766 - accuracy: 0.3902
4/4 [==============================] - 7s 1s/step - loss: 1.6018 - accuracy: 0.2946
1/1 [==============================] - 0s 485ms/step - loss: 1.6066 - accuracy: 0.2195
4/4 [==============================] - 6s 967ms/step - loss: 1.6062 - accuracy: 0.2892
1/1 [==============================] - 1s 508ms/step - loss: 1.5894 - accuracy: 0.3415
4/4 [==============================] - 7s 1s/step - loss: 1.6017 - accuracy: 0.3243
1/1 [==============================] - 1s 515ms/step - loss: 1.5797 - accuracy: 0.4390
4/4 [==============================] - 6s 991ms/step - loss: 1.6055 - accuracy: 0.2351
1/1 [==============================] - 1s 797ms/step - loss: 1.5901 - accuracy: 0.3171
4/4 [==============================] - 6s 1s/step - loss: 1.6016 - accuracy: 0.2405
1/1 [==============================] - 0s 490ms/step - loss: 1.5966 - accuracy: 0.2927
4/4 [==============================] - 7s 2s/step - loss: 1.6026 - accuracy: 0.2676
1/1 [==============================] - 1s 502ms/step - loss: 1.6097 - accuracy: 0.2927
4/4 [==============================] - 7s 1s/step - loss: 1.6043 - accuracy: 0.2757
1/1 [==============================] - 1s 746ms/step - loss: 1.5898 - accuracy: 0.3659
4/4 [==============================] - 2s 93ms/step - loss: 1.6130 - accuracy: 0.2005
1/1 [==============================] - 0s 319ms/step - loss: 1.6097 - accuracy: 0.2381
4/4 [==============================] - 2s 91ms/step - loss: 1.6006 - accuracy: 0.2378
1/1 [==============================] - 0s 318ms/step - loss: 1.6010 - accuracy: 0.1707
4/4 [==============================] - 2s 88ms/step - loss: 1.6002 - accuracy: 0.2324
1/1 [==============================] - 0s 310ms/step - loss: 1.5910 - accuracy: 0.2195
4/4 [==============================] - 2s 133ms/step - loss: 1.6049 - accuracy: 0.2486
1/1 [==============================] - 0s 453ms/step - loss: 1.6070 - accuracy: 0.1220
4/4 [==============================] - 4s 100ms/step - loss: 1.6095 - accuracy: 0.2216
1/1 [==============================] - 0s 350ms/step - loss: 1.6018 - accuracy: 0.3415
4/4 [==============================] - 2s 96ms/step - loss: 1.6087 - accuracy: 0.2649
1/1 [==============================] - 0s 342ms/step - loss: 1.6082 - accuracy: 0.1951
4/4 [==============================] - 2s 94ms/step - loss: 1.6060 - accuracy: 0.3378
1/1 [==============================] - 0s 333ms/step - loss: 1.6017 - accuracy: 0.3171
4/4 [==============================] - 3s 145ms/step - loss: 1.6088 - accuracy: 0.2351
1/1 [==============================] - 1s 518ms/step - loss: 1.6082 - accuracy: 0.2927
4/4 [==============================] - 2s 92ms/step - loss: 1.6089 - accuracy: 0.2514
1/1 [==============================] - 0s 317ms/step - loss: 1.6116 - accuracy: 0.1220
4/4 [==============================] - 2s 106ms/step - loss: 1.6035 - accuracy: 0.2703
1/1 [==============================] - 0s 467ms/step - loss: 1.6032 - accuracy: 0.1463
4/4 [==============================] - 3s 95ms/step - loss: 1.6101 - accuracy: 0.1545
1/1 [==============================] - 0s 321ms/step - loss: 1.6039 - accuracy: 0.3571
4/4 [==============================] - 3s 150ms/step - loss: 1.6027 - accuracy: 0.3351
1/1 [==============================] - 1s 509ms/step - loss: 1.5995 - accuracy: 0.3415
4/4 [==============================] - 2s 97ms/step - loss: 1.6002 - accuracy: 0.2324
1/1 [==============================] - 0s 335ms/step - loss: 1.5904 - accuracy: 0.2195
4/4 [==============================] - 2s 97ms/step - loss: 1.6007 - accuracy: 0.2568
1/1 [==============================] - 0s 331ms/step - loss: 1.6059 - accuracy: 0.2683
4/4 [==============================] - 2s 96ms/step - loss: 1.6059 - accuracy: 0.3000
1/1 [==============================] - 0s 305ms/step - loss: 1.6039 - accuracy: 0.3415
4/4 [==============================] - 2s 151ms/step - loss: 1.6059 - accuracy: 0.2270
1/1 [==============================] - 0s 477ms/step - loss: 1.6006 - accuracy: 0.2683
4/4 [==============================] - 2s 97ms/step - loss: 1.6066 - accuracy: 0.2568
1/1 [==============================] - 2s 2s/step - loss: 1.6023 - accuracy: 0.2683
4/4 [==============================] - 2s 97ms/step - loss: 1.6023 - accuracy: 0.2405
1/1 [==============================] - 0s 332ms/step - loss: 1.6002 - accuracy: 0.1463
4/4 [==============================] - 2s 103ms/step - loss: 1.6101 - accuracy: 0.2432
1/1 [==============================] - 0s 326ms/step - loss: 1.6104 - accuracy: 0.1220
4/4 [==============================] - 3s 146ms/step - loss: 1.6116 - accuracy: 0.1541
1/1 [==============================] - 0s 476ms/step - loss: 1.6092 - accuracy: 0.1463
4/4 [==============================] - 2s 114ms/step - loss: 1.6073 - accuracy: 0.2547
1/1 [==============================] - 0s 338ms/step - loss: 1.5986 - accuracy: 0.3571
4/4 [==============================] - 2s 118ms/step - loss: 1.6100 - accuracy: 0.2459
1/1 [==============================] - 0s 326ms/step - loss: 1.6024 - accuracy: 0.3659
4/4 [==============================] - 2s 121ms/step - loss: 1.6056 - accuracy: 0.2324
1/1 [==============================] - 0s 317ms/step - loss: 1.5971 - accuracy: 0.2195
4/4 [==============================] - 2s 127ms/step - loss: 1.6085 - accuracy: 0.2946
1/1 [==============================] - 0s 488ms/step - loss: 1.6074 - accuracy: 0.2195
4/4 [==============================] - 3s 113ms/step - loss: 1.6061 - accuracy: 0.2595
1/1 [==============================] - 0s 335ms/step - loss: 1.5991 - accuracy: 0.2439
4/4 [==============================] - 2s 112ms/step - loss: 1.6087 - accuracy: 0.2568
1/1 [==============================] - 0s 324ms/step - loss: 1.5960 - accuracy: 0.4390
4/4 [==============================] - 2s 117ms/step - loss: 1.6098 - accuracy: 0.2568
1/1 [==============================] - 0s 303ms/step - loss: 1.6001 - accuracy: 0.2683
4/4 [==============================] - 2s 120ms/step - loss: 1.6037 - accuracy: 0.2405
1/1 [==============================] - 0s 312ms/step - loss: 1.6037 - accuracy: 0.1463
4/4 [==============================] - 3s 191ms/step - loss: 1.6103 - accuracy: 0.2432
1/1 [==============================] - 1s 509ms/step - loss: 1.6143 - accuracy: 0.1220
4/4 [==============================] - 2s 117ms/step - loss: 1.6087 - accuracy: 0.2189
1/1 [==============================] - 0s 346ms/step - loss: 1.6007 - accuracy: 0.3415
4/4 [==============================] - 3s 211ms/step - loss: 1.6089 - accuracy: 0.1924
1/1 [==============================] - 0s 332ms/step - loss: 1.6017 - accuracy: 0.3571
4/4 [==============================] - 3s 257ms/step - loss: 1.6009 - accuracy: 0.3351
1/1 [==============================] - 0s 477ms/step - loss: 1.5895 - accuracy: 0.3415
4/4 [==============================] - 3s 211ms/step - loss: 1.6010 - accuracy: 0.2297
1/1 [==============================] - 0s 339ms/step - loss: 1.5859 - accuracy: 0.3902
4/4 [==============================] - 2s 206ms/step - loss: 1.6024 - accuracy: 0.3486
1/1 [==============================] - 0s 342ms/step - loss: 1.6105 - accuracy: 0.2195
4/4 [==============================] - 2s 208ms/step - loss: 1.6039 - accuracy: 0.2027
1/1 [==============================] - 0s 333ms/step - loss: 1.5851 - accuracy: 0.3415
4/4 [==============================] - 3s 372ms/step - loss: 1.6090 - accuracy: 0.1568
1/1 [==============================] - 0s 326ms/step - loss: 1.6003 - accuracy: 0.4390
4/4 [==============================] - 2s 208ms/step - loss: 1.6035 - accuracy: 0.3135
1/1 [==============================] - 0s 332ms/step - loss: 1.5966 - accuracy: 0.3171
4/4 [==============================] - 2s 211ms/step - loss: 1.6068 - accuracy: 0.2216
1/1 [==============================] - 0s 341ms/step - loss: 1.6029 - accuracy: 0.2927
4/4 [==============================] - 2s 213ms/step - loss: 1.6069 - accuracy: 0.1784
1/1 [==============================] - 0s 478ms/step - loss: 1.6109 - accuracy: 0.2927
4/4 [==============================] - 3s 212ms/step - loss: 1.6070 - accuracy: 0.2189
1/1 [==============================] - 0s 333ms/step - loss: 1.5988 - accuracy: 0.1463
4/4 [==============================] - 5s 439ms/step - loss: 1.6062 - accuracy: 0.2683
1/1 [==============================] - 0s 433ms/step - loss: 1.5979 - accuracy: 0.3571
4/4 [==============================] - 4s 700ms/step - loss: 1.6049 - accuracy: 0.3351
1/1 [==============================] - 0s 410ms/step - loss: 1.5994 - accuracy: 0.3415
4/4 [==============================] - 3s 447ms/step - loss: 1.6054 - accuracy: 0.3297
1/1 [==============================] - 0s 469ms/step - loss: 1.5952 - accuracy: 0.3902
4/4 [==============================] - 4s 492ms/step - loss: 1.6064 - accuracy: 0.2784
1/1 [==============================] - 1s 622ms/step - loss: 1.6076 - accuracy: 0.2195
4/4 [==============================] - 4s 451ms/step - loss: 1.6063 - accuracy: 0.2946
1/1 [==============================] - 0s 416ms/step - loss: 1.5970 - accuracy: 0.3415
4/4 [==============================] - 3s 441ms/step - loss: 1.6049 - accuracy: 0.2649
1/1 [==============================] - 0s 414ms/step - loss: 1.5939 - accuracy: 0.1951
4/4 [==============================] - 4s 600ms/step - loss: 1.6075 - accuracy: 0.2703
1/1 [==============================] - 1s 583ms/step - loss: 1.6002 - accuracy: 0.3171
4/4 [==============================] - 4s 429ms/step - loss: 1.6042 - accuracy: 0.2405
1/1 [==============================] - 0s 399ms/step - loss: 1.6037 - accuracy: 0.1463
4/4 [==============================] - 3s 428ms/step - loss: 1.5983 - accuracy: 0.3405
1/1 [==============================] - 0s 381ms/step - loss: 1.6076 - accuracy: 0.2927
4/4 [==============================] - 5s 624ms/step - loss: 1.6065 - accuracy: 0.1919
1/1 [==============================] - 0s 377ms/step - loss: 1.5949 - accuracy: 0.3415
4/4 [==============================] - 6s 993ms/step - loss: 1.6077 - accuracy: 0.2304
1/1 [==============================] - 1s 512ms/step - loss: 1.5956 - accuracy: 0.2381
4/4 [==============================] - 7s 1s/step - loss: 1.6000 - accuracy: 0.2946
1/1 [==============================] - 0s 481ms/step - loss: 1.5809 - accuracy: 0.3415
4/4 [==============================] - 9s 2s/step - loss: 1.6019 - accuracy: 0.2946
1/1 [==============================] - 2s 2s/step - loss: 1.5803 - accuracy: 0.3902
4/4 [==============================] - 6s 987ms/step - loss: 1.6004 - accuracy: 0.3486
1/1 [==============================] - 1s 834ms/step - loss: 1.6060 - accuracy: 0.2195
4/4 [==============================] - 6s 998ms/step - loss: 1.6054 - accuracy: 0.2946
1/1 [==============================] - 1s 507ms/step - loss: 1.5889 - accuracy: 0.3415
4/4 [==============================] - 7s 2s/step - loss: 1.6013 - accuracy: 0.2270
1/1 [==============================] - 0s 495ms/step - loss: 1.5793 - accuracy: 0.4390
4/4 [==============================] - 7s 1s/step - loss: 1.6058 - accuracy: 0.1730
1/1 [==============================] - 1s 849ms/step - loss: 1.5918 - accuracy: 0.3171
4/4 [==============================] - 6s 1s/step - loss: 1.6014 - accuracy: 0.3405
1/1 [==============================] - 1s 523ms/step - loss: 1.5930 - accuracy: 0.2927
4/4 [==============================] - 7s 2s/step - loss: 1.6048 - accuracy: 0.2405
1/1 [==============================] - 1s 517ms/step - loss: 1.6095 - accuracy: 0.2927
4/4 [==============================] - 7s 2s/step - loss: 1.6019 - accuracy: 0.2622
1/1 [==============================] - 0s 494ms/step - loss: 1.5863 - accuracy: 0.3659
19/19 [==============================] - 3s 80ms/step - loss: 1.6079 - accuracy: 0.2412
3/3 [==============================] - 0s 16ms/step - loss: 1.6023 - accuracy: 0.3571
19/19 [==============================] - 4s 115ms/step - loss: 1.6014 - accuracy: 0.3351
3/3 [==============================] - 0s 14ms/step - loss: 1.6037 - accuracy: 0.3415
19/19 [==============================] - 3s 77ms/step - loss: 1.6062 - accuracy: 0.3054
3/3 [==============================] - 0s 16ms/step - loss: 1.5958 - accuracy: 0.3902
19/19 [==============================] - 3s 79ms/step - loss: 1.6010 - accuracy: 0.3000
3/3 [==============================] - 0s 16ms/step - loss: 1.6150 - accuracy: 0.1220
19/19 [==============================] - 4s 113ms/step - loss: 1.6086 - accuracy: 0.2730
3/3 [==============================] - 0s 15ms/step - loss: 1.6000 - accuracy: 0.3415
19/19 [==============================] - 3s 79ms/step - loss: 1.6046 - accuracy: 0.2459
3/3 [==============================] - 0s 16ms/step - loss: 1.5914 - accuracy: 0.1951
19/19 [==============================] - 3s 78ms/step - loss: 1.6017 - accuracy: 0.2351
3/3 [==============================] - 0s 15ms/step - loss: 1.5863 - accuracy: 0.2683
19/19 [==============================] - 4s 98ms/step - loss: 1.6107 - accuracy: 0.2486
3/3 [==============================] - 0s 15ms/step - loss: 1.6001 - accuracy: 0.2927
19/19 [==============================] - 3s 78ms/step - loss: 1.6065 - accuracy: 0.2108
3/3 [==============================] - 0s 15ms/step - loss: 1.6100 - accuracy: 0.1220
19/19 [==============================] - 3s 92ms/step - loss: 1.5998 - accuracy: 0.3189
3/3 [==============================] - 1s 21ms/step - loss: 1.5830 - accuracy: 0.3659
19/19 [==============================] - 3s 83ms/step - loss: 1.6006 - accuracy: 0.2575
3/3 [==============================] - 0s 16ms/step - loss: 1.5906 - accuracy: 0.2619
19/19 [==============================] - 3s 82ms/step - loss: 1.5943 - accuracy: 0.2459
3/3 [==============================] - 0s 17ms/step - loss: 1.5627 - accuracy: 0.3659
19/19 [==============================] - 4s 119ms/step - loss: 1.6022 - accuracy: 0.2243
3/3 [==============================] - 1s 22ms/step - loss: 1.5850 - accuracy: 0.2195
19/19 [==============================] - 3s 82ms/step - loss: 1.5875 - accuracy: 0.3486
3/3 [==============================] - 0s 22ms/step - loss: 1.6060 - accuracy: 0.2195
19/19 [==============================] - 3s 84ms/step - loss: 1.6008 - accuracy: 0.2892
3/3 [==============================] - 0s 16ms/step - loss: 1.5760 - accuracy: 0.3415
19/19 [==============================] - 4s 124ms/step - loss: 1.5996 - accuracy: 0.2757
3/3 [==============================] - 0s 16ms/step - loss: 1.5844 - accuracy: 0.4390
19/19 [==============================] - 3s 82ms/step - loss: 1.5989 - accuracy: 0.2243
3/3 [==============================] - 0s 16ms/step - loss: 1.5735 - accuracy: 0.3171
19/19 [==============================] - 3s 84ms/step - loss: 1.5960 - accuracy: 0.2378
3/3 [==============================] - 0s 27ms/step - loss: 1.5881 - accuracy: 0.3415
19/19 [==============================] - 4s 82ms/step - loss: 1.6013 - accuracy: 0.3351
3/3 [==============================] - 0s 15ms/step - loss: 1.6056 - accuracy: 0.2927
19/19 [==============================] - 3s 83ms/step - loss: 1.6018 - accuracy: 0.2595
3/3 [==============================] - 0s 17ms/step - loss: 1.6010 - accuracy: 0.1463
19/19 [==============================] - 5s 142ms/step - loss: 1.5951 - accuracy: 0.3008
3/3 [==============================] - 0s 18ms/step - loss: 1.5703 - accuracy: 0.2381
19/19 [==============================] - 3s 88ms/step - loss: 1.5996 - accuracy: 0.3000
3/3 [==============================] - 0s 17ms/step - loss: 1.5858 - accuracy: 0.3415
19/19 [==============================] - 3s 87ms/step - loss: 1.5985 - accuracy: 0.2135
3/3 [==============================] - 0s 18ms/step - loss: 1.5704 - accuracy: 0.2195
19/19 [==============================] - 4s 105ms/step - loss: 1.5924 - accuracy: 0.3189
3/3 [==============================] - 0s 18ms/step - loss: 1.5983 - accuracy: 0.2195
19/19 [==============================] - 3s 91ms/step - loss: 1.5897 - accuracy: 0.3108
3/3 [==============================] - 0s 18ms/step - loss: 1.5457 - accuracy: 0.3415
19/19 [==============================] - 3s 89ms/step - loss: 1.6022 - accuracy: 0.2919
3/3 [==============================] - 1s 30ms/step - loss: 1.5837 - accuracy: 0.4390
19/19 [==============================] - 4s 89ms/step - loss: 1.6008 - accuracy: 0.3189
3/3 [==============================] - 0s 18ms/step - loss: 1.5841 - accuracy: 0.3171
19/19 [==============================] - 3s 87ms/step - loss: 1.5937 - accuracy: 0.3081
3/3 [==============================] - 0s 19ms/step - loss: 1.5897 - accuracy: 0.2927
19/19 [==============================] - 3s 101ms/step - loss: 1.5924 - accuracy: 0.2784
3/3 [==============================] - 1s 31ms/step - loss: 1.6095 - accuracy: 0.2927
19/19 [==============================] - 3s 91ms/step - loss: 1.5938 - accuracy: 0.3324
3/3 [==============================] - 0s 18ms/step - loss: 1.5569 - accuracy: 0.3659
19/19 [==============================] - 4s 105ms/step - loss: 1.5925 - accuracy: 0.2981
3/3 [==============================] - 0s 21ms/step - loss: 1.5572 - accuracy: 0.3571
19/19 [==============================] - 5s 136ms/step - loss: 1.5928 - accuracy: 0.2892
3/3 [==============================] - 0s 21ms/step - loss: 1.5747 - accuracy: 0.3415
19/19 [==============================] - 4s 104ms/step - loss: 1.5995 - accuracy: 0.2514
3/3 [==============================] - 0s 20ms/step - loss: 1.5753 - accuracy: 0.3171
19/19 [==============================] - 4s 112ms/step - loss: 1.5964 - accuracy: 0.3054
3/3 [==============================] - 1s 27ms/step - loss: 1.6069 - accuracy: 0.2195
19/19 [==============================] - 4s 105ms/step - loss: 1.5936 - accuracy: 0.3000
3/3 [==============================] - 0s 20ms/step - loss: 1.5563 - accuracy: 0.3415
19/19 [==============================] - 4s 104ms/step - loss: 1.5933 - accuracy: 0.2054
3/3 [==============================] - 0s 20ms/step - loss: 1.5596 - accuracy: 0.4390
19/19 [==============================] - 5s 159ms/step - loss: 1.5878 - accuracy: 0.2000
3/3 [==============================] - 0s 20ms/step - loss: 1.5058 - accuracy: 0.3171
19/19 [==============================] - 4s 104ms/step - loss: 1.5927 - accuracy: 0.2730
3/3 [==============================] - 0s 20ms/step - loss: 1.5874 - accuracy: 0.2927
19/19 [==============================] - 4s 114ms/step - loss: 1.5827 - accuracy: 0.2676
3/3 [==============================] - 1s 29ms/step - loss: 1.6144 - accuracy: 0.2927
19/19 [==============================] - 4s 103ms/step - loss: 1.6032 - accuracy: 0.2568
3/3 [==============================] - 0s 24ms/step - loss: 1.5889 - accuracy: 0.3659
19/19 [==============================] - 7s 199ms/step - loss: 1.5912 - accuracy: 0.2791
3/3 [==============================] - 1s 71ms/step - loss: 1.5520 - accuracy: 0.3571
19/19 [==============================] - 6s 204ms/step - loss: 1.5693 - accuracy: 0.2946
3/3 [==============================] - 0s 40ms/step - loss: 1.4835 - accuracy: 0.3415
19/19 [==============================] - 7s 291ms/step - loss: 1.5718 - accuracy: 0.3297
3/3 [==============================] - 0s 39ms/step - loss: 1.3862 - accuracy: 0.3902
19/19 [==============================] - 6s 207ms/step - loss: 1.5711 - accuracy: 0.3270
3/3 [==============================] - 0s 43ms/step - loss: 6915550609408.0000 - accuracy: 0.2195
19/19 [==============================] - 7s 280ms/step - loss: 1.5674 - accuracy: 0.3243
3/3 [==============================] - 0s 42ms/step - loss: 1.4332 - accuracy: 0.3415
19/19 [==============================] - 5s 203ms/step - loss: 1.5692 - accuracy: 0.3243
3/3 [==============================] - 0s 42ms/step - loss: 1.3689 - accuracy: 0.4390
19/19 [==============================] - 7s 262ms/step - loss: 1.5789 - accuracy: 0.3189
3/3 [==============================] - 0s 42ms/step - loss: 1.4543 - accuracy: 0.3171
19/19 [==============================] - 5s 204ms/step - loss: 1.5895 - accuracy: 0.2649
3/3 [==============================] - 0s 43ms/step - loss: 1.5596 - accuracy: 0.2927
19/19 [==============================] - 7s 241ms/step - loss: 194102984704.0000 - accuracy: 0.3270
3/3 [==============================] - 0s 44ms/step - loss: 1.7086 - accuracy: 0.2927
19/19 [==============================] - 5s 205ms/step - loss: 1.5826 - accuracy: 0.3189
3/3 [==============================] - 0s 42ms/step - loss: 1.5199 - accuracy: 0.3659
19/19 [==============================] - 13s 532ms/step - loss: 251981873152.0000 - accuracy: 0.2737
3/3 [==============================] - 1s 86ms/step - loss: 1.5279 - accuracy: 0.3571
19/19 [==============================] - 11s 481ms/step - loss: 13798.5605 - accuracy: 0.3189
3/3 [==============================] - 1s 76ms/step - loss: 1.5304 - accuracy: 0.3415
19/19 [==============================] - 12s 523ms/step - loss: nan - accuracy: 0.3027
3/3 [==============================] - 2s 72ms/step - loss: nan - accuracy: 0.0488
19/19 [==============================] - 12s 518ms/step - loss: nan - accuracy: 0.3027
3/3 [==============================] - 1s 76ms/step - loss: nan - accuracy: 0.2683
19/19 [==============================] - 12s 542ms/step - loss: 2005213904896.0000 - accuracy: 0.3162
3/3 [==============================] - 1s 76ms/step - loss: 1.4479 - accuracy: 0.3415
19/19 [==============================] - 12s 538ms/step - loss: nan - accuracy: 0.2811
3/3 [==============================] - 1s 77ms/step - loss: nan - accuracy: 0.0732
19/19 [==============================] - 12s 535ms/step - loss: 1.5549 - accuracy: 0.3270
3/3 [==============================] - 1s 74ms/step - loss: 1.5189 - accuracy: 0.3171
19/19 [==============================] - 12s 536ms/step - loss: 1407724032.0000 - accuracy: 0.3405
3/3 [==============================] - 1s 79ms/step - loss: 1.5649 - accuracy: 0.2927
19/19 [==============================] - 12s 544ms/step - loss: 31469.2266 - accuracy: 0.2946
3/3 [==============================] - 1s 73ms/step - loss: 1.6179 - accuracy: 0.2927
19/19 [==============================] - 11s 494ms/step - loss: nan - accuracy: 0.2811
3/3 [==============================] - 1s 117ms/step - loss: nan - accuracy: 0.0732
19/19 [==============================] - 3s 80ms/step - loss: 1.6021 - accuracy: 0.2412
3/3 [==============================] - 0s 16ms/step - loss: 1.5844 - accuracy: 0.3571
19/19 [==============================] - 3s 81ms/step - loss: 1.5976 - accuracy: 0.3351
3/3 [==============================] - 0s 18ms/step - loss: 1.5861 - accuracy: 0.3415
19/19 [==============================] - 4s 113ms/step - loss: 1.6059 - accuracy: 0.2838
3/3 [==============================] - 0s 15ms/step - loss: 1.5952 - accuracy: 0.3902
19/19 [==============================] - 3s 81ms/step - loss: 1.6000 - accuracy: 0.3486
3/3 [==============================] - 0s 15ms/step - loss: 1.6091 - accuracy: 0.2195
19/19 [==============================] - 3s 80ms/step - loss: 1.6006 - accuracy: 0.3216
3/3 [==============================] - 0s 16ms/step - loss: 1.5708 - accuracy: 0.3415
19/19 [==============================] - 5s 79ms/step - loss: 1.5960 - accuracy: 0.2514
3/3 [==============================] - 0s 14ms/step - loss: 1.5669 - accuracy: 0.1951
19/19 [==============================] - 3s 82ms/step - loss: 1.6033 - accuracy: 0.3378
3/3 [==============================] - 0s 18ms/step - loss: 1.5979 - accuracy: 0.3171
19/19 [==============================] - 3s 84ms/step - loss: 1.5842 - accuracy: 0.3405
3/3 [==============================] - 0s 22ms/step - loss: 1.5775 - accuracy: 0.2927
19/19 [==============================] - 4s 115ms/step - loss: 1.5995 - accuracy: 0.3270
3/3 [==============================] - 1s 20ms/step - loss: 1.6087 - accuracy: 0.2927
19/19 [==============================] - 3s 81ms/step - loss: 1.5921 - accuracy: 0.3270
3/3 [==============================] - 0s 17ms/step - loss: 1.5776 - accuracy: 0.3659
19/19 [==============================] - 4s 126ms/step - loss: 1.6006 - accuracy: 0.2005
3/3 [==============================] - 0s 23ms/step - loss: 1.5816 - accuracy: 0.2381
19/19 [==============================] - 3s 85ms/step - loss: 1.5954 - accuracy: 0.3351
3/3 [==============================] - 0s 19ms/step - loss: 1.5747 - accuracy: 0.3415
19/19 [==============================] - 3s 83ms/step - loss: 1.6035 - accuracy: 0.3297
3/3 [==============================] - 0s 16ms/step - loss: 1.5879 - accuracy: 0.3902
19/19 [==============================] - 4s 119ms/step - loss: 1.6014 - accuracy: 0.2432
3/3 [==============================] - 0s 19ms/step - loss: 1.6151 - accuracy: 0.1220
19/19 [==============================] - 3s 83ms/step - loss: 1.5994 - accuracy: 0.2541
3/3 [==============================] - 0s 16ms/step - loss: 1.5838 - accuracy: 0.3415
19/19 [==============================] - 3s 83ms/step - loss: 1.6025 - accuracy: 0.2649
3/3 [==============================] - 0s 16ms/step - loss: 1.5892 - accuracy: 0.4390
19/19 [==============================] - 4s 118ms/step - loss: 1.5994 - accuracy: 0.3378
3/3 [==============================] - 1s 17ms/step - loss: 1.5869 - accuracy: 0.3171
19/19 [==============================] - 3s 84ms/step - loss: 1.6076 - accuracy: 0.2405
3/3 [==============================] - 0s 16ms/step - loss: 1.6031 - accuracy: 0.1463
19/19 [==============================] - 3s 83ms/step - loss: 1.5939 - accuracy: 0.3297
3/3 [==============================] - 1s 23ms/step - loss: 1.6083 - accuracy: 0.2927
19/19 [==============================] - 4s 83ms/step - loss: 1.6031 - accuracy: 0.2189
3/3 [==============================] - 0s 17ms/step - loss: 1.5954 - accuracy: 0.3415
19/19 [==============================] - 3s 91ms/step - loss: 1.6003 - accuracy: 0.2168
3/3 [==============================] - 0s 18ms/step - loss: 1.5863 - accuracy: 0.2619
19/19 [==============================] - 4s 110ms/step - loss: 1.5957 - accuracy: 0.3108
3/3 [==============================] - 1s 29ms/step - loss: 1.5792 - accuracy: 0.3415
19/19 [==============================] - 4s 90ms/step - loss: 1.5887 - accuracy: 0.3297
3/3 [==============================] - 0s 21ms/step - loss: 1.5475 - accuracy: 0.3902
19/19 [==============================] - 3s 93ms/step - loss: 1.5873 - accuracy: 0.3378
3/3 [==============================] - 0s 19ms/step - loss: 1.6160 - accuracy: 0.2195
19/19 [==============================] - 5s 134ms/step - loss: 1.5852 - accuracy: 0.3351
3/3 [==============================] - 0s 17ms/step - loss: 1.5345 - accuracy: 0.3415
19/19 [==============================] - 3s 92ms/step - loss: 1.5884 - accuracy: 0.3243
3/3 [==============================] - 0s 18ms/step - loss: 1.5380 - accuracy: 0.4390
19/19 [==============================] - 3s 89ms/step - loss: 1.6021 - accuracy: 0.1892
3/3 [==============================] - 0s 18ms/step - loss: 1.5839 - accuracy: 0.2683
19/19 [==============================] - 4s 131ms/step - loss: 1.5978 - accuracy: 0.2486
3/3 [==============================] - 0s 19ms/step - loss: 1.5843 - accuracy: 0.3415
19/19 [==============================] - 3s 89ms/step - loss: 1.5876 - accuracy: 0.3000
3/3 [==============================] - 0s 17ms/step - loss: 1.6115 - accuracy: 0.2927
19/19 [==============================] - 3s 92ms/step - loss: 1.5922 - accuracy: 0.3324
3/3 [==============================] - 1s 29ms/step - loss: 1.5706 - accuracy: 0.3659
19/19 [==============================] - 4s 103ms/step - loss: 1.5841 - accuracy: 0.3333
3/3 [==============================] - 0s 22ms/step - loss: 1.5407 - accuracy: 0.3571
19/19 [==============================] - 4s 104ms/step - loss: 1.5903 - accuracy: 0.3351
3/3 [==============================] - 0s 20ms/step - loss: 1.5712 - accuracy: 0.3415
19/19 [==============================] - 4s 124ms/step - loss: 1.6003 - accuracy: 0.3189
3/3 [==============================] - 1s 31ms/step - loss: 1.5783 - accuracy: 0.3902
19/19 [==============================] - 4s 105ms/step - loss: 1.5798 - accuracy: 0.3486
3/3 [==============================] - 0s 19ms/step - loss: 1.6089 - accuracy: 0.2195
19/19 [==============================] - 4s 104ms/step - loss: 1.5946 - accuracy: 0.3000
3/3 [==============================] - 0s 20ms/step - loss: 1.5526 - accuracy: 0.3415
19/19 [==============================] - 5s 160ms/step - loss: 1.5892 - accuracy: 0.2189
3/3 [==============================] - 0s 22ms/step - loss: 1.5382 - accuracy: 0.4390
19/19 [==============================] - 4s 106ms/step - loss: 1.5903 - accuracy: 0.3243
3/3 [==============================] - 0s 20ms/step - loss: 1.5621 - accuracy: 0.3171
19/19 [==============================] - 4s 138ms/step - loss: 1.5952 - accuracy: 0.2270
3/3 [==============================] - 0s 21ms/step - loss: 1.5903 - accuracy: 0.1463
19/19 [==============================] - 4s 105ms/step - loss: 1.5836 - accuracy: 0.3297
3/3 [==============================] - 0s 25ms/step - loss: 1.6082 - accuracy: 0.2927
19/19 [==============================] - 4s 123ms/step - loss: 1.5807 - accuracy: 0.3270
3/3 [==============================] - 1s 36ms/step - loss: 1.5408 - accuracy: 0.3659
19/19 [==============================] - 6s 206ms/step - loss: 1.5779 - accuracy: 0.3144
3/3 [==============================] - 0s 49ms/step - loss: 1.4916 - accuracy: 0.3571
19/19 [==============================] - 7s 306ms/step - loss: 1.5934 - accuracy: 0.3108
3/3 [==============================] - 0s 46ms/step - loss: 1.5658 - accuracy: 0.3415
19/19 [==============================] - 9s 326ms/step - loss: 1.5935 - accuracy: 0.3027
3/3 [==============================] - 0s 44ms/step - loss: 1.5432 - accuracy: 0.3902
19/19 [==============================] - 6s 240ms/step - loss: 145087.8125 - accuracy: 0.3486
3/3 [==============================] - 1s 70ms/step - loss: 1.6347 - accuracy: 0.2195
19/19 [==============================] - 6s 213ms/step - loss: 1.5846 - accuracy: 0.3162
3/3 [==============================] - 0s 43ms/step - loss: 1.5165 - accuracy: 0.3415
19/19 [==============================] - 7s 286ms/step - loss: 1.5962 - accuracy: 0.2892
3/3 [==============================] - 0s 43ms/step - loss: 1.5566 - accuracy: 0.4390
19/19 [==============================] - 5s 203ms/step - loss: 1.5801 - accuracy: 0.2243
3/3 [==============================] - 0s 44ms/step - loss: 1.4426 - accuracy: 0.3171
19/19 [==============================] - 7s 296ms/step - loss: 1.5843 - accuracy: 0.2486
3/3 [==============================] - 0s 44ms/step - loss: 1.5672 - accuracy: 0.2927
19/19 [==============================] - 5s 203ms/step - loss: 1.5852 - accuracy: 0.3216
3/3 [==============================] - 0s 48ms/step - loss: 1.6151 - accuracy: 0.2927
19/19 [==============================] - 7s 291ms/step - loss: 1.5780 - accuracy: 0.3189
3/3 [==============================] - 0s 44ms/step - loss: 1350145024.0000 - accuracy: 0.3659
19/19 [==============================] - 12s 549ms/step - loss: 706805.5000 - accuracy: 0.2683
3/3 [==============================] - 1s 133ms/step - loss: 1.4696 - accuracy: 0.3571
19/19 [==============================] - 11s 457ms/step - loss: nan - accuracy: 0.3243
3/3 [==============================] - 1s 111ms/step - loss: nan - accuracy: 0.0244
19/19 [==============================] - 10s 454ms/step - loss: nan - accuracy: 0.2730
3/3 [==============================] - 2s 77ms/step - loss: nan - accuracy: 0.0488
19/19 [==============================] - 12s 542ms/step - loss: 14976219611136.0000 - accuracy: 0.3486
3/3 [==============================] - 1s 77ms/step - loss: 1.6138 - accuracy: 0.2195
19/19 [==============================] - 12s 526ms/step - loss: nan - accuracy: 0.3135
3/3 [==============================] - 1s 80ms/step - loss: nan - accuracy: 0.0488
19/19 [==============================] - 11s 509ms/step - loss: 1.5626 - accuracy: 0.3243
3/3 [==============================] - 1s 118ms/step - loss: 1.5170 - accuracy: 0.4390
19/19 [==============================] - 11s 486ms/step - loss: nan - accuracy: 0.3135
3/3 [==============================] - 1s 136ms/step - loss: nan - accuracy: 0.0732
19/19 [==============================] - 11s 488ms/step - loss: 754206720.0000 - accuracy: 0.3027
3/3 [==============================] - 1s 137ms/step - loss: 1.5605 - accuracy: 0.2927
19/19 [==============================] - 12s 472ms/step - loss: nan - accuracy: 0.2216
3/3 [==============================] - 1s 110ms/step - loss: nan - accuracy: 0.2439
19/19 [==============================] - 11s 468ms/step - loss: 1.5626 - accuracy: 0.3270
3/3 [==============================] - 1s 130ms/step - loss: 1.5392 - accuracy: 0.3659
8/8 [==============================] - 3s 87ms/step - loss: 1.6021 - accuracy: 0.2981
1/1 [==============================] - 0s 348ms/step - loss: 1.5881 - accuracy: 0.2619
8/8 [==============================] - 2s 86ms/step - loss: 1.6087 - accuracy: 0.2541
1/1 [==============================] - 0s 350ms/step - loss: 1.6058 - accuracy: 0.3415
8/8 [==============================] - 2s 86ms/step - loss: 1.6061 - accuracy: 0.3297
1/1 [==============================] - 0s 301ms/step - loss: 1.6017 - accuracy: 0.3902
8/8 [==============================] - 3s 128ms/step - loss: 1.6125 - accuracy: 0.0649
1/1 [==============================] - 0s 449ms/step - loss: 1.6164 - accuracy: 0.1220
8/8 [==============================] - 3s 88ms/step - loss: 1.6001 - accuracy: 0.3351
1/1 [==============================] - 0s 313ms/step - loss: 1.5899 - accuracy: 0.3415
8/8 [==============================] - 2s 88ms/step - loss: 1.6068 - accuracy: 0.2108
1/1 [==============================] - 0s 327ms/step - loss: 1.6001 - accuracy: 0.2683
8/8 [==============================] - 2s 88ms/step - loss: 1.6030 - accuracy: 0.3162
1/1 [==============================] - 0s 324ms/step - loss: 1.6036 - accuracy: 0.3171
8/8 [==============================] - 3s 130ms/step - loss: 1.6027 - accuracy: 0.3135
1/1 [==============================] - 0s 481ms/step - loss: 1.6055 - accuracy: 0.2927
8/8 [==============================] - 2s 88ms/step - loss: 1.6080 - accuracy: 0.3135
1/1 [==============================] - 0s 351ms/step - loss: 1.6097 - accuracy: 0.2927
8/8 [==============================] - 2s 85ms/step - loss: 1.6009 - accuracy: 0.3324
1/1 [==============================] - 0s 339ms/step - loss: 1.5920 - accuracy: 0.3659
8/8 [==============================] - 2s 91ms/step - loss: 1.6083 - accuracy: 0.2033
1/1 [==============================] - 0s 425ms/step - loss: 1.5994 - accuracy: 0.2619
8/8 [==============================] - 3s 93ms/step - loss: 1.6085 - accuracy: 0.2162
1/1 [==============================] - 0s 342ms/step - loss: 1.6023 - accuracy: 0.3659
8/8 [==============================] - 2s 88ms/step - loss: 1.6102 - accuracy: 0.1838
1/1 [==============================] - 0s 314ms/step - loss: 1.6064 - accuracy: 0.2195
8/8 [==============================] - 2s 95ms/step - loss: 1.6035 - accuracy: 0.3027
1/1 [==============================] - 0s 309ms/step - loss: 1.6065 - accuracy: 0.2195
8/8 [==============================] - 2s 106ms/step - loss: 1.6252 - accuracy: 0.0784
1/1 [==============================] - 0s 438ms/step - loss: 1.6122 - accuracy: 0.2439
8/8 [==============================] - 3s 94ms/step - loss: 1.6050 - accuracy: 0.1568
1/1 [==============================] - 0s 313ms/step - loss: 1.5949 - accuracy: 0.2683
8/8 [==============================] - 4s 91ms/step - loss: 1.6001 - accuracy: 0.3216
1/1 [==============================] - 0s 343ms/step - loss: 1.5885 - accuracy: 0.3171
8/8 [==============================] - 2s 92ms/step - loss: 1.6098 - accuracy: 0.1784
1/1 [==============================] - 0s 321ms/step - loss: 1.5992 - accuracy: 0.3415
8/8 [==============================] - 3s 139ms/step - loss: 1.6001 - accuracy: 0.3405
1/1 [==============================] - 0s 472ms/step - loss: 1.6109 - accuracy: 0.2927
8/8 [==============================] - 2s 94ms/step - loss: 1.5978 - accuracy: 0.3324
1/1 [==============================] - 0s 340ms/step - loss: 1.5717 - accuracy: 0.3659
8/8 [==============================] - 2s 101ms/step - loss: 1.6054 - accuracy: 0.2981
1/1 [==============================] - 0s 348ms/step - loss: 1.5956 - accuracy: 0.3571
8/8 [==============================] - 2s 101ms/step - loss: 1.6028 - accuracy: 0.3351
1/1 [==============================] - 0s 326ms/step - loss: 1.5951 - accuracy: 0.3415
8/8 [==============================] - 3s 157ms/step - loss: 1.6053 - accuracy: 0.2514
1/1 [==============================] - 0s 326ms/step - loss: 1.5899 - accuracy: 0.3171
8/8 [==============================] - 2s 100ms/step - loss: 1.6089 - accuracy: 0.2216
1/1 [==============================] - 0s 317ms/step - loss: 1.6115 - accuracy: 0.1220
8/8 [==============================] - 2s 97ms/step - loss: 1.6022 - accuracy: 0.3351
1/1 [==============================] - 0s 311ms/step - loss: 1.5878 - accuracy: 0.3415
8/8 [==============================] - 2s 100ms/step - loss: 1.6057 - accuracy: 0.2946
1/1 [==============================] - 0s 311ms/step - loss: 1.5952 - accuracy: 0.4390
8/8 [==============================] - 3s 133ms/step - loss: 1.6035 - accuracy: 0.3378
1/1 [==============================] - 0s 315ms/step - loss: 1.5978 - accuracy: 0.3171
8/8 [==============================] - 2s 98ms/step - loss: 1.6092 - accuracy: 0.1459
1/1 [==============================] - 0s 308ms/step - loss: 1.6088 - accuracy: 0.1463
8/8 [==============================] - 2s 97ms/step - loss: 1.6010 - accuracy: 0.2757
1/1 [==============================] - 1s 1s/step - loss: 1.6145 - accuracy: 0.1707
8/8 [==============================] - 3s 155ms/step - loss: 1.5893 - accuracy: 0.3324
1/1 [==============================] - 1s 528ms/step - loss: 1.5675 - accuracy: 0.3659
8/8 [==============================] - 3s 131ms/step - loss: 1.6080 - accuracy: 0.2168
1/1 [==============================] - 0s 342ms/step - loss: 1.5969 - accuracy: 0.2381
8/8 [==============================] - 3s 129ms/step - loss: 1.5929 - accuracy: 0.3351
1/1 [==============================] - 0s 354ms/step - loss: 1.5775 - accuracy: 0.3415
8/8 [==============================] - 3s 130ms/step - loss: 1.6056 - accuracy: 0.2162
1/1 [==============================] - 0s 333ms/step - loss: 1.5905 - accuracy: 0.3171
8/8 [==============================] - 4s 164ms/step - loss: 1.6036 - accuracy: 0.3000
1/1 [==============================] - 0s 315ms/step - loss: 1.6055 - accuracy: 0.2195
8/8 [==============================] - 3s 131ms/step - loss: 1.6048 - accuracy: 0.2757
1/1 [==============================] - 0s 344ms/step - loss: 1.5866 - accuracy: 0.3415
8/8 [==============================] - 3s 127ms/step - loss: 1.6057 - accuracy: 0.2000
1/1 [==============================] - 0s 328ms/step - loss: 1.5947 - accuracy: 0.2683
8/8 [==============================] - 4s 203ms/step - loss: 1.6012 - accuracy: 0.3378
1/1 [==============================] - 0s 341ms/step - loss: 1.5840 - accuracy: 0.3171
8/8 [==============================] - 3s 129ms/step - loss: 1.6024 - accuracy: 0.2189
1/1 [==============================] - 0s 328ms/step - loss: 1.5973 - accuracy: 0.3415
8/8 [==============================] - 3s 129ms/step - loss: 1.6007 - accuracy: 0.2865
1/1 [==============================] - 0s 332ms/step - loss: 1.6080 - accuracy: 0.1707
8/8 [==============================] - 3s 169ms/step - loss: 1.5992 - accuracy: 0.3324
1/1 [==============================] - 0s 462ms/step - loss: 1.5816 - accuracy: 0.3659
8/8 [==============================] - 6s 486ms/step - loss: 1.5996 - accuracy: 0.2629
1/1 [==============================] - 1s 650ms/step - loss: 1.5754 - accuracy: 0.3571
8/8 [==============================] - 5s 397ms/step - loss: 1.5999 - accuracy: 0.2703
1/1 [==============================] - 0s 417ms/step - loss: 1.5796 - accuracy: 0.3415
8/8 [==============================] - 4s 302ms/step - loss: 1.5978 - accuracy: 0.3297
1/1 [==============================] - 0s 401ms/step - loss: 1.5738 - accuracy: 0.3902
8/8 [==============================] - 4s 359ms/step - loss: 1.5969 - accuracy: 0.3108
1/1 [==============================] - 1s 609ms/step - loss: 1.6069 - accuracy: 0.2195
8/8 [==============================] - 5s 315ms/step - loss: 1.5974 - accuracy: 0.3189
1/1 [==============================] - 0s 387ms/step - loss: 1.5661 - accuracy: 0.3415
8/8 [==============================] - 4s 314ms/step - loss: 1.5979 - accuracy: 0.3243
1/1 [==============================] - 0s 404ms/step - loss: 1.5689 - accuracy: 0.4390
8/8 [==============================] - 6s 460ms/step - loss: 1.5974 - accuracy: 0.2081
1/1 [==============================] - 0s 379ms/step - loss: 1.5696 - accuracy: 0.3171
8/8 [==============================] - 4s 308ms/step - loss: 1.5994 - accuracy: 0.2595
1/1 [==============================] - 0s 405ms/step - loss: 1.5964 - accuracy: 0.2927
8/8 [==============================] - 5s 389ms/step - loss: 1.5971 - accuracy: 0.3135
1/1 [==============================] - 1s 600ms/step - loss: 1.6056 - accuracy: 0.2927
8/8 [==============================] - 4s 311ms/step - loss: 1.5996 - accuracy: 0.3135
1/1 [==============================] - 0s 417ms/step - loss: 1.5861 - accuracy: 0.3659
8/8 [==============================] - 8s 783ms/step - loss: 1.5995 - accuracy: 0.2656
1/1 [==============================] - 0s 476ms/step - loss: 1.5726 - accuracy: 0.3571
8/8 [==============================] - 8s 748ms/step - loss: 1.5964 - accuracy: 0.2757
1/1 [==============================] - 1s 786ms/step - loss: 1.5698 - accuracy: 0.3415
8/8 [==============================] - 7s 677ms/step - loss: 1.5944 - accuracy: 0.3135
1/1 [==============================] - 1s 548ms/step - loss: 1.5492 - accuracy: 0.3902
8/8 [==============================] - 9s 787ms/step - loss: 1.5857 - accuracy: 0.3162
1/1 [==============================] - 0s 488ms/step - loss: 1.6160 - accuracy: 0.2195
8/8 [==============================] - 9s 888ms/step - loss: 1.5888 - accuracy: 0.3189
1/1 [==============================] - 1s 581ms/step - loss: 1.5217 - accuracy: 0.3415
8/8 [==============================] - 7s 688ms/step - loss: 1.5939 - accuracy: 0.3027
1/1 [==============================] - 1s 503ms/step - loss: 1.5457 - accuracy: 0.4390
8/8 [==============================] - 9s 773ms/step - loss: 1.6002 - accuracy: 0.3054
1/1 [==============================] - 0s 486ms/step - loss: 1.5739 - accuracy: 0.3171
8/8 [==============================] - 9s 799ms/step - loss: 1.5976 - accuracy: 0.2811
1/1 [==============================] - 0s 499ms/step - loss: 1.5878 - accuracy: 0.2927
8/8 [==============================] - 8s 820ms/step - loss: 1.5960 - accuracy: 0.2297
1/1 [==============================] - 1s 503ms/step - loss: 1.6204 - accuracy: 0.1220
8/8 [==============================] - 8s 776ms/step - loss: 1.5948 - accuracy: 0.3000
1/1 [==============================] - 0s 500ms/step - loss: 1.5510 - accuracy: 0.3659
8/8 [==============================] - 2s 85ms/step - loss: 1.6100 - accuracy: 0.1626
1/1 [==============================] - 0s 308ms/step - loss: 1.6056 - accuracy: 0.3571
8/8 [==============================] - 2s 86ms/step - loss: 1.5931 - accuracy: 0.2946
1/1 [==============================] - 0s 429ms/step - loss: 1.5750 - accuracy: 0.3415
8/8 [==============================] - 3s 94ms/step - loss: 1.5921 - accuracy: 0.3297
1/1 [==============================] - 0s 314ms/step - loss: 1.5710 - accuracy: 0.3902
8/8 [==============================] - 2s 85ms/step - loss: 1.6039 - accuracy: 0.3108
1/1 [==============================] - 0s 322ms/step - loss: 1.6105 - accuracy: 0.2195
8/8 [==============================] - 2s 83ms/step - loss: 1.6056 - accuracy: 0.2514
1/1 [==============================] - 2s 2s/step - loss: 1.5943 - accuracy: 0.2439
8/8 [==============================] - 4s 140ms/step - loss: 1.5886 - accuracy: 0.3243
1/1 [==============================] - 1s 558ms/step - loss: 1.5569 - accuracy: 0.4390
8/8 [==============================] - 3s 89ms/step - loss: 1.6067 - accuracy: 0.2216
1/1 [==============================] - 0s 323ms/step - loss: 1.5981 - accuracy: 0.3171
8/8 [==============================] - 2s 90ms/step - loss: 1.5984 - accuracy: 0.2405
1/1 [==============================] - 0s 327ms/step - loss: 1.5984 - accuracy: 0.1463
8/8 [==============================] - 2s 88ms/step - loss: 1.6062 - accuracy: 0.2432
1/1 [==============================] - 0s 376ms/step - loss: 1.6107 - accuracy: 0.1220
8/8 [==============================] - 3s 103ms/step - loss: 1.6096 - accuracy: 0.2676
1/1 [==============================] - 0s 312ms/step - loss: 1.6034 - accuracy: 0.3659
8/8 [==============================] - 2s 94ms/step - loss: 1.5991 - accuracy: 0.2981
1/1 [==============================] - 0s 312ms/step - loss: 1.5828 - accuracy: 0.3571
8/8 [==============================] - 2s 92ms/step - loss: 1.6043 - accuracy: 0.3351
1/1 [==============================] - 0s 321ms/step - loss: 1.5990 - accuracy: 0.3415
8/8 [==============================] - 2s 95ms/step - loss: 1.6081 - accuracy: 0.2919
1/1 [==============================] - 0s 468ms/step - loss: 1.5990 - accuracy: 0.3902
8/8 [==============================] - 3s 91ms/step - loss: 1.5960 - accuracy: 0.3351
1/1 [==============================] - 0s 339ms/step - loss: 1.6098 - accuracy: 0.2195
8/8 [==============================] - 2s 91ms/step - loss: 1.5986 - accuracy: 0.3351
1/1 [==============================] - 0s 325ms/step - loss: 1.5837 - accuracy: 0.3415
8/8 [==============================] - 2s 93ms/step - loss: 1.6057 - accuracy: 0.3243
1/1 [==============================] - 0s 314ms/step - loss: 1.5972 - accuracy: 0.4390
8/8 [==============================] - 3s 133ms/step - loss: 1.6032 - accuracy: 0.2216
1/1 [==============================] - 1s 518ms/step - loss: 1.5865 - accuracy: 0.3171
8/8 [==============================] - 2s 93ms/step - loss: 1.6022 - accuracy: 0.2405
1/1 [==============================] - 0s 343ms/step - loss: 1.6036 - accuracy: 0.1463
8/8 [==============================] - 2s 89ms/step - loss: 1.5961 - accuracy: 0.2459
1/1 [==============================] - 0s 335ms/step - loss: 1.6087 - accuracy: 0.2927
8/8 [==============================] - 3s 133ms/step - loss: 1.6070 - accuracy: 0.1162
1/1 [==============================] - 1s 508ms/step - loss: 1.6010 - accuracy: 0.3659
8/8 [==============================] - 3s 100ms/step - loss: 1.5991 - accuracy: 0.3333
1/1 [==============================] - 0s 322ms/step - loss: 1.5864 - accuracy: 0.3571
8/8 [==============================] - 2s 105ms/step - loss: 1.5999 - accuracy: 0.3351
1/1 [==============================] - 0s 320ms/step - loss: 1.5933 - accuracy: 0.3415
8/8 [==============================] - 2s 101ms/step - loss: 1.6025 - accuracy: 0.2919
1/1 [==============================] - 0s 316ms/step - loss: 1.5890 - accuracy: 0.2195
8/8 [==============================] - 3s 157ms/step - loss: 1.5936 - accuracy: 0.3486
1/1 [==============================] - 0s 403ms/step - loss: 1.6067 - accuracy: 0.2195
8/8 [==============================] - 2s 103ms/step - loss: 1.5970 - accuracy: 0.3351
1/1 [==============================] - 0s 307ms/step - loss: 1.5781 - accuracy: 0.3415
8/8 [==============================] - 2s 99ms/step - loss: 1.5997 - accuracy: 0.2270
1/1 [==============================] - 0s 311ms/step - loss: 1.5808 - accuracy: 0.2683
8/8 [==============================] - 2s 99ms/step - loss: 1.6084 - accuracy: 0.2216
1/1 [==============================] - 0s 310ms/step - loss: 1.5988 - accuracy: 0.3171
8/8 [==============================] - 3s 144ms/step - loss: 1.6001 - accuracy: 0.3243
1/1 [==============================] - 0s 308ms/step - loss: 1.5942 - accuracy: 0.2927
8/8 [==============================] - 2s 98ms/step - loss: 1.6062 - accuracy: 0.1324
1/1 [==============================] - 1s 1s/step - loss: 1.6080 - accuracy: 0.1220
8/8 [==============================] - 2s 98ms/step - loss: 1.6015 - accuracy: 0.2189
1/1 [==============================] - 0s 323ms/step - loss: 1.5773 - accuracy: 0.3415
8/8 [==============================] - 4s 202ms/step - loss: 1.5932 - accuracy: 0.2304
1/1 [==============================] - 0s 327ms/step - loss: 1.5735 - accuracy: 0.2381
8/8 [==============================] - 3s 132ms/step - loss: 1.6022 - accuracy: 0.2919
1/1 [==============================] - 0s 366ms/step - loss: 1.5881 - accuracy: 0.3415
8/8 [==============================] - 3s 130ms/step - loss: 1.6030 - accuracy: 0.3297
1/1 [==============================] - 0s 315ms/step - loss: 1.5899 - accuracy: 0.3902
8/8 [==============================] - 3s 176ms/step - loss: 1.5973 - accuracy: 0.2568
1/1 [==============================] - 1s 515ms/step - loss: 1.6053 - accuracy: 0.2683
8/8 [==============================] - 3s 132ms/step - loss: 1.6027 - accuracy: 0.3108
1/1 [==============================] - 0s 327ms/step - loss: 1.5891 - accuracy: 0.3415
8/8 [==============================] - 3s 131ms/step - loss: 1.6045 - accuracy: 0.2081
1/1 [==============================] - 0s 335ms/step - loss: 1.5890 - accuracy: 0.1951
8/8 [==============================] - 3s 134ms/step - loss: 1.6037 - accuracy: 0.1730
1/1 [==============================] - 0s 413ms/step - loss: 1.5891 - accuracy: 0.3171
8/8 [==============================] - 4s 142ms/step - loss: 1.5984 - accuracy: 0.2486
1/1 [==============================] - 0s 324ms/step - loss: 1.5858 - accuracy: 0.3415
8/8 [==============================] - 3s 130ms/step - loss: 1.5938 - accuracy: 0.3405
1/1 [==============================] - 0s 333ms/step - loss: 1.6055 - accuracy: 0.2927
8/8 [==============================] - 3s 129ms/step - loss: 1.6062 - accuracy: 0.2270
1/1 [==============================] - 0s 342ms/step - loss: 1.5939 - accuracy: 0.3659
8/8 [==============================] - 6s 490ms/step - loss: 1.6002 - accuracy: 0.2602
1/1 [==============================] - 0s 394ms/step - loss: 1.5801 - accuracy: 0.3571
8/8 [==============================] - 4s 315ms/step - loss: 1.6028 - accuracy: 0.2649
1/1 [==============================] - 0s 408ms/step - loss: 1.5940 - accuracy: 0.3415
8/8 [==============================] - 6s 513ms/step - loss: 1.5988 - accuracy: 0.3297
1/1 [==============================] - 0s 399ms/step - loss: 1.5743 - accuracy: 0.3902
8/8 [==============================] - 4s 320ms/step - loss: 1.5996 - accuracy: 0.3108
1/1 [==============================] - 0s 391ms/step - loss: 1.6047 - accuracy: 0.2195
8/8 [==============================] - 5s 371ms/step - loss: 1.5969 - accuracy: 0.3351
1/1 [==============================] - 1s 599ms/step - loss: 1.5714 - accuracy: 0.3415
8/8 [==============================] - 5s 318ms/step - loss: 1.5996 - accuracy: 0.2919
1/1 [==============================] - 0s 401ms/step - loss: 1.5743 - accuracy: 0.4390
8/8 [==============================] - 4s 306ms/step - loss: 1.6024 - accuracy: 0.2270
1/1 [==============================] - 0s 410ms/step - loss: 1.5838 - accuracy: 0.3171
8/8 [==============================] - 6s 508ms/step - loss: 1.5991 - accuracy: 0.2405
1/1 [==============================] - 1s 658ms/step - loss: 1.5909 - accuracy: 0.1463
8/8 [==============================] - 4s 303ms/step - loss: 1.5984 - accuracy: 0.2162
1/1 [==============================] - 0s 389ms/step - loss: 1.6129 - accuracy: 0.2927
8/8 [==============================] - 6s 455ms/step - loss: 1.5963 - accuracy: 0.3324
1/1 [==============================] - 0s 414ms/step - loss: 1.5735 - accuracy: 0.3659
8/8 [==============================] - 7s 711ms/step - loss: 1.5972 - accuracy: 0.2737
1/1 [==============================] - 0s 476ms/step - loss: 1.5624 - accuracy: 0.3571
8/8 [==============================] - 7s 685ms/step - loss: 1.5936 - accuracy: 0.2946
1/1 [==============================] - 1s 527ms/step - loss: 1.5563 - accuracy: 0.3415
8/8 [==============================] - 8s 676ms/step - loss: 1.5941 - accuracy: 0.3135
1/1 [==============================] - 2s 2s/step - loss: 1.5419 - accuracy: 0.3902
8/8 [==============================] - 9s 881ms/step - loss: 1.5897 - accuracy: 0.3108
1/1 [==============================] - 1s 526ms/step - loss: 1.6090 - accuracy: 0.2195
8/8 [==============================] - 7s 700ms/step - loss: 1.5911 - accuracy: 0.2973
1/1 [==============================] - 1s 787ms/step - loss: 1.5241 - accuracy: 0.3415
8/8 [==============================] - 8s 674ms/step - loss: 1.5955 - accuracy: 0.2919
1/1 [==============================] - 1s 776ms/step - loss: 1.5575 - accuracy: 0.4390
8/8 [==============================] - 7s 676ms/step - loss: 1.5944 - accuracy: 0.3351
1/1 [==============================] - 1s 826ms/step - loss: 1.5496 - accuracy: 0.3171
8/8 [==============================] - 7s 676ms/step - loss: 1.5915 - accuracy: 0.3378
1/1 [==============================] - 0s 492ms/step - loss: 1.5771 - accuracy: 0.2927
8/8 [==============================] - 7s 684ms/step - loss: 1.5937 - accuracy: 0.2378
1/1 [==============================] - 0s 499ms/step - loss: 1.6155 - accuracy: 0.2927
8/8 [==============================] - 7s 681ms/step - loss: 1.5848 - accuracy: 0.3135
1/1 [==============================] - 1s 517ms/step - loss: 1.5214 - accuracy: 0.3659
5/5 [==============================] - 6s 903ms/step - loss: 1.6017 - accuracy: 0.2579
best parameters for ANN:  {'batch_size': 100, 'nb_epoch': 20, 'neurons': 256}
best score for ANN: 0.3357142835855484

Best parameters of tuning in Word2vec-

best parameters for ANN: {'batch_size': 100, 'nb_epoch': 20, 'neurons': 256}

best score for ANN: 0.3357142835855484

##Function with tuned parameters

def LSTM_Model_Tuned_WV (X_train, X_test, y_train, y_test):
  in_dim = X_train.shape[1]

  tf.random.set_seed(7)
  embedding_vecor_length = 32

  model = Sequential()
  model.add(Embedding(max_features, embedding_vecor_length, input_length=in_dim))
  model.add(SpatialDropout1D(0.2))
  model.add(LSTM(256))
  model.add(Dense(5 , activation='softmax'))

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=0, patience=3)

  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

  print(model.summary())

  history=model.fit(X_train, y_train_cat, validation_split=0.2, epochs = 20, batch_size = 100, callbacks=[early_stopping])

  # train_score = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  # test_score = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  # result_kfold_df= pd.DataFrame({'model': ['Neural Network'], 'train accuracy': [train_score], 'test accuracy': [test_score] })

  train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  result_kfold_df= pd.DataFrame({'model': ['LSTM'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })


  # print(result_kfold_df)

  hist= pd.DataFrame(history.history)

  for col in hist.columns:
    print(col)
    plt.plot(hist[col])
    plt.plot(hist[col])
    plt.title('model-'+col)
    plt.ylabel(col)
    plt.xlabel('epoch')
    plt.show()

  return result_kfold_df

LSTM_Model_Tuned_WV(X_train_wv, X_test_wv, y_train_wv, y_test_wv)

Model: "sequential_1882"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1882 (Embedding)  (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_75 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1881 (LSTM)            (None, 256)               295936    
                                                                 
 dense_1881 (Dense)          (None, 5)                 1285      
                                                                 
=================================================================
Total params: 303,621
Trainable params: 303,621
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 10s 2s/step - loss: 1.5987 - accuracy: 0.2443 - val_loss: 1.5740 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 6s 2s/step - loss: 1.5268 - accuracy: 0.3359 - val_loss: 1.8823 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 5s 2s/step - loss: 1.5527 - accuracy: 0.3359 - val_loss: 1.5398 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 4s 1s/step - loss: 1.4904 - accuracy: 0.3321 - val_loss: 1.5471 - val_accuracy: 0.3333
Epoch 5/20
3/3 [==============================] - 4s 1s/step - loss: 1.4909 - accuracy: 0.3244 - val_loss: 1.5476 - val_accuracy: 0.2121
Epoch 6/20
3/3 [==============================] - 3s 1s/step - loss: 1.4753 - accuracy: 0.3550 - val_loss: 1.5576 - val_accuracy: 0.3333
11/11 [==============================] - 2s 163ms/step
3/3 [==============================] - 0s 144ms/step
11/11 [==============================] - 2s 167ms/step
3/3 [==============================] - 1s 241ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.335366	0.337349	0.168449	0.170194

3 times run-

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model_Tuned_WV(X_train_wv, X_test_wv, y_train_wv, y_test_wv)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_1883"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1883 (Embedding)  (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_76 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1882 (LSTM)            (None, 256)               295936    
                                                                 
 dense_1882 (Dense)          (None, 5)                 1285      
                                                                 
=================================================================
Total params: 303,621
Trainable params: 303,621
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 6s 1s/step - loss: 1.5970 - accuracy: 0.2863 - val_loss: 1.5737 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 4s 2s/step - loss: 1.5164 - accuracy: 0.3359 - val_loss: 1.5648 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 4s 1s/step - loss: 1.4723 - accuracy: 0.3244 - val_loss: 1.5675 - val_accuracy: 0.2273
Epoch 4/20
3/3 [==============================] - 3s 1s/step - loss: 1.4759 - accuracy: 0.2481 - val_loss: 1.5571 - val_accuracy: 0.3333
Epoch 5/20
3/3 [==============================] - 3s 1s/step - loss: 1.4615 - accuracy: 0.3359 - val_loss: 1.5550 - val_accuracy: 0.3333
Epoch 6/20
3/3 [==============================] - 5s 2s/step - loss: 1.4616 - accuracy: 0.3359 - val_loss: 1.5627 - val_accuracy: 0.3333
Epoch 7/20
3/3 [==============================] - 3s 1s/step - loss: 1.4625 - accuracy: 0.3359 - val_loss: 1.5663 - val_accuracy: 0.3333
Epoch 8/20
3/3 [==============================] - 3s 1s/step - loss: 1.4656 - accuracy: 0.3359 - val_loss: 1.5674 - val_accuracy: 0.3333
11/11 [==============================] - 4s 212ms/step
3/3 [==============================] - 1s 329ms/step
11/11 [==============================] - 4s 340ms/step
3/3 [==============================] - 1s 153ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_1884"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1884 (Embedding)  (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_77 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1883 (LSTM)            (None, 256)               295936    
                                                                 
 dense_1883 (Dense)          (None, 5)                 1285      
                                                                 
=================================================================
Total params: 303,621
Trainable params: 303,621
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 7s 2s/step - loss: 1.5975 - accuracy: 0.2786 - val_loss: 1.5750 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 5s 1s/step - loss: 1.5197 - accuracy: 0.3359 - val_loss: 2.0618 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 3s 1s/step - loss: 1.6098 - accuracy: 0.3359 - val_loss: 1.5428 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 3s 1s/step - loss: 1.4935 - accuracy: 0.2824 - val_loss: 1.5533 - val_accuracy: 0.2121
Epoch 5/20
3/3 [==============================] - 5s 2s/step - loss: 1.5028 - accuracy: 0.2710 - val_loss: 1.5536 - val_accuracy: 0.2121
Epoch 6/20
3/3 [==============================] - 4s 1s/step - loss: 1.4922 - accuracy: 0.2710 - val_loss: 1.5555 - val_accuracy: 0.2121
11/11 [==============================] - 2s 163ms/step
3/3 [==============================] - 1s 223ms/step
11/11 [==============================] - 2s 180ms/step
3/3 [==============================] - 0s 148ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_1885"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1885 (Embedding)  (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_78 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1884 (LSTM)            (None, 256)               295936    
                                                                 
 dense_1884 (Dense)          (None, 5)                 1285      
                                                                 
=================================================================
Total params: 303,621
Trainable params: 303,621
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 7s 2s/step - loss: 1.6031 - accuracy: 0.2214 - val_loss: 1.5764 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 4s 1s/step - loss: 1.5318 - accuracy: 0.3359 - val_loss: 1.7459 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 3s 1s/step - loss: 1.5213 - accuracy: 0.3359 - val_loss: 1.5383 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 3s 1s/step - loss: 1.4939 - accuracy: 0.3359 - val_loss: 1.5427 - val_accuracy: 0.3333
Epoch 5/20
3/3 [==============================] - 4s 2s/step - loss: 1.4918 - accuracy: 0.3359 - val_loss: 1.5409 - val_accuracy: 0.3333
Epoch 6/20
3/3 [==============================] - 4s 1s/step - loss: 1.4751 - accuracy: 0.3359 - val_loss: 1.5478 - val_accuracy: 0.3333
11/11 [==============================] - 3s 193ms/step
3/3 [==============================] - 1s 252ms/step
11/11 [==============================] - 3s 229ms/step
3/3 [==============================] - 0s 156ms/step
loss

accuracy

val_loss

val_accuracy

Result of all runs:   model  train accuracy  test accuracy  train F1 score  test F1 score
0  LSTM        0.335366       0.337349        0.168449       0.170194
1  LSTM        0.259146       0.253012        0.106670       0.102178
2  LSTM        0.335366       0.337349        0.168449       0.170194

Smote Word2vec Dataset-

LSTM_Model_Tuned_WV(X_train_wv_smote, X_test_wv, y_train_wv_smote, y_test_wv)

Model: "sequential_1886"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1886 (Embedding)  (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_79 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1885 (LSTM)            (None, 256)               295936    
                                                                 
 dense_1885 (Dense)          (None, 5)                 1285      
                                                                 
=================================================================
Total params: 303,621
Trainable params: 303,621
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 10s 2s/step - loss: 1.5968 - accuracy: 0.2295 - val_loss: 1.9354 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 6s 1s/step - loss: 1.5875 - accuracy: 0.2591 - val_loss: 2.2254 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 7s 1s/step - loss: 1.5357 - accuracy: 0.2545 - val_loss: 2.0254 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 6s 1s/step - loss: 1.5417 - accuracy: 0.2500 - val_loss: 2.1564 - val_accuracy: 0.0000e+00
18/18 [==============================] - 5s 217ms/step
3/3 [==============================] - 0s 144ms/step
18/18 [==============================] - 3s 163ms/step
3/3 [==============================] - 0s 143ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.2	0.228916	0.066667	0.085282

3 times run

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model_Tuned_WV(X_train_wv_smote, X_test_wv, y_train_wv_smote, y_test_wv)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_1887"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1887 (Embedding)  (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_80 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1886 (LSTM)            (None, 256)               295936    
                                                                 
 dense_1886 (Dense)          (None, 5)                 1285      
                                                                 
=================================================================
Total params: 303,621
Trainable params: 303,621
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 10s 1s/step - loss: 1.5957 - accuracy: 0.2000 - val_loss: 1.9452 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 8s 2s/step - loss: 1.5849 - accuracy: 0.2432 - val_loss: 2.1280 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 6s 1s/step - loss: 1.5426 - accuracy: 0.2432 - val_loss: 1.9551 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 6s 1s/step - loss: 1.5488 - accuracy: 0.2159 - val_loss: 2.0405 - val_accuracy: 0.0000e+00
18/18 [==============================] - 4s 181ms/step
3/3 [==============================] - 0s 135ms/step
18/18 [==============================] - 3s 164ms/step
3/3 [==============================] - 1s 158ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_1888"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1888 (Embedding)  (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_81 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1887 (LSTM)            (None, 256)               295936    
                                                                 
 dense_1887 (Dense)          (None, 5)                 1285      
                                                                 
=================================================================
Total params: 303,621
Trainable params: 303,621
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 9s 1s/step - loss: 1.5897 - accuracy: 0.2432 - val_loss: 2.3480 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 7s 1s/step - loss: 1.5699 - accuracy: 0.2182 - val_loss: 2.1189 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 5s 1s/step - loss: 1.5413 - accuracy: 0.2568 - val_loss: 1.9856 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 7s 1s/step - loss: 1.5441 - accuracy: 0.2500 - val_loss: 2.1151 - val_accuracy: 0.0000e+00
Epoch 5/20
5/5 [==============================] - 6s 1s/step - loss: 1.5339 - accuracy: 0.2500 - val_loss: 2.4987 - val_accuracy: 0.0000e+00
Epoch 6/20
5/5 [==============================] - 6s 1s/step - loss: 1.5245 - accuracy: 0.2318 - val_loss: 2.8232 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 165ms/step
3/3 [==============================] - 0s 143ms/step
18/18 [==============================] - 4s 220ms/step
3/3 [==============================] - 0s 143ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_1889"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1889 (Embedding)  (None, 200, 32)           6400      
                                                                 
 spatial_dropout1d_82 (Spati  (None, 200, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1888 (LSTM)            (None, 256)               295936    
                                                                 
 dense_1888 (Dense)          (None, 5)                 1285      
                                                                 
=================================================================
Total params: 303,621
Trainable params: 303,621
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 11s 2s/step - loss: 1.5950 - accuracy: 0.2409 - val_loss: 1.9608 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 6s 1s/step - loss: 1.6139 - accuracy: 0.2432 - val_loss: 2.3240 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 7s 1s/step - loss: 1.5348 - accuracy: 0.2500 - val_loss: 2.0330 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 6s 1s/step - loss: 1.5437 - accuracy: 0.2295 - val_loss: 2.1199 - val_accuracy: 0.0000e+00
18/18 [==============================] - 5s 252ms/step
3/3 [==============================] - 1s 150ms/step
18/18 [==============================] - 3s 169ms/step
3/3 [==============================] - 1s 156ms/step
loss

accuracy

val_loss

val_accuracy

Result of all runs:   model  train accuracy  test accuracy  train F1 score  test F1 score
0  LSTM             0.2       0.228916        0.066667       0.085282
1  LSTM             0.2       0.337349        0.066667       0.170194
2  LSTM             0.2       0.228916        0.066667       0.085282

Word2vec Full dataset-

LSTM_Model_Tuned_WV(X_train_wvfull, X_test_wvfull, y_train_wvfull, y_test_wvfull)

Model: "sequential_1890"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1890 (Embedding)  (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_83 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1889 (LSTM)            (None, 256)               295936    
                                                                 
 dense_1889 (Dense)          (None, 5)                 1285      
                                                                 
=================================================================
Total params: 303,621
Trainable params: 303,621
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 8s 2s/step - loss: 1.6015 - accuracy: 0.2405 - val_loss: 1.5808 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 4s 1s/step - loss: 1.5379 - accuracy: 0.3359 - val_loss: 1.7601 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 4s 1s/step - loss: 1.5196 - accuracy: 0.3359 - val_loss: 1.5414 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 4s 1s/step - loss: 1.4908 - accuracy: 0.2748 - val_loss: 1.5474 - val_accuracy: 0.2121
Epoch 5/20
3/3 [==============================] - 4s 1s/step - loss: 1.4823 - accuracy: 0.2710 - val_loss: 1.5518 - val_accuracy: 0.2121
Epoch 6/20
3/3 [==============================] - 4s 1s/step - loss: 1.4662 - accuracy: 0.2710 - val_loss: 1.5749 - val_accuracy: 0.2121
11/11 [==============================] - 2s 181ms/step
3/3 [==============================] - 1s 155ms/step
11/11 [==============================] - 2s 189ms/step
3/3 [==============================] - 1s 162ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.259146	0.253012	0.10667	0.102178

3 times run-

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model_Tuned_WV(X_train_wvfull, X_test_wvfull, y_train_wvfull, y_test_wvfull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_1891"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1891 (Embedding)  (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_84 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1890 (LSTM)            (None, 256)               295936    
                                                                 
 dense_1890 (Dense)          (None, 5)                 1285      
                                                                 
=================================================================
Total params: 303,621
Trainable params: 303,621
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 8s 2s/step - loss: 1.5968 - accuracy: 0.2748 - val_loss: 1.5747 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 4s 1s/step - loss: 1.5242 - accuracy: 0.3359 - val_loss: 1.9288 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 5s 2s/step - loss: 1.5733 - accuracy: 0.3359 - val_loss: 1.5418 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 4s 1s/step - loss: 1.4943 - accuracy: 0.3321 - val_loss: 1.5475 - val_accuracy: 0.3333
Epoch 5/20
3/3 [==============================] - 4s 1s/step - loss: 1.4940 - accuracy: 0.3321 - val_loss: 1.5457 - val_accuracy: 0.3333
Epoch 6/20
3/3 [==============================] - 4s 2s/step - loss: 1.4762 - accuracy: 0.3359 - val_loss: 1.5585 - val_accuracy: 0.3333
11/11 [==============================] - 2s 179ms/step
3/3 [==============================] - 1s 155ms/step
11/11 [==============================] - 2s 174ms/step
3/3 [==============================] - 0s 151ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_1892"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1892 (Embedding)  (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_85 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1891 (LSTM)            (None, 256)               295936    
                                                                 
 dense_1891 (Dense)          (None, 5)                 1285      
                                                                 
=================================================================
Total params: 303,621
Trainable params: 303,621
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 8s 2s/step - loss: 1.5973 - accuracy: 0.2443 - val_loss: 1.5715 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 4s 1s/step - loss: 1.5198 - accuracy: 0.3359 - val_loss: 1.5735 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 5s 2s/step - loss: 1.4768 - accuracy: 0.3359 - val_loss: 1.5457 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 4s 1s/step - loss: 1.4741 - accuracy: 0.2824 - val_loss: 1.5804 - val_accuracy: 0.3333
Epoch 5/20
3/3 [==============================] - 4s 1s/step - loss: 1.4648 - accuracy: 0.3397 - val_loss: 1.5667 - val_accuracy: 0.3333
Epoch 6/20
3/3 [==============================] - 4s 1s/step - loss: 1.4687 - accuracy: 0.3359 - val_loss: 1.5470 - val_accuracy: 0.3333
11/11 [==============================] - 2s 175ms/step
3/3 [==============================] - 1s 165ms/step
11/11 [==============================] - 2s 184ms/step
3/3 [==============================] - 1s 163ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_1893"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1893 (Embedding)  (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_86 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1892 (LSTM)            (None, 256)               295936    
                                                                 
 dense_1892 (Dense)          (None, 5)                 1285      
                                                                 
=================================================================
Total params: 303,621
Trainable params: 303,621
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 7s 2s/step - loss: 1.5975 - accuracy: 0.3015 - val_loss: 1.5746 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 4s 1s/step - loss: 1.5303 - accuracy: 0.3359 - val_loss: 1.9475 - val_accuracy: 0.3333
Epoch 3/20
3/3 [==============================] - 5s 2s/step - loss: 1.5791 - accuracy: 0.3359 - val_loss: 1.5442 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 4s 1s/step - loss: 1.4954 - accuracy: 0.2977 - val_loss: 1.5593 - val_accuracy: 0.2121
Epoch 5/20
3/3 [==============================] - 4s 1s/step - loss: 1.4985 - accuracy: 0.2710 - val_loss: 1.5653 - val_accuracy: 0.2121
Epoch 6/20
3/3 [==============================] - 5s 2s/step - loss: 1.4867 - accuracy: 0.2710 - val_loss: 1.5741 - val_accuracy: 0.2121
11/11 [==============================] - 3s 181ms/step
3/3 [==============================] - 1s 165ms/step
11/11 [==============================] - 2s 177ms/step
3/3 [==============================] - 1s 153ms/step
loss

accuracy

val_loss

val_accuracy

Result of all runs:   model  train accuracy  test accuracy  train F1 score  test F1 score
0  LSTM        0.259146       0.253012        0.106670       0.102178
1  LSTM        0.335366       0.337349        0.168449       0.170194
2  LSTM        0.335366       0.337349        0.168449       0.170194

Word2vec full smote dataset-

LSTM_Model_Tuned_WV(X_train_wvfull_smote, X_test_wvfull, y_train_wvfull_smote, y_test_wvfull)

Model: "sequential_1894"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1894 (Embedding)  (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_87 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1893 (LSTM)            (None, 256)               295936    
                                                                 
 dense_1893 (Dense)          (None, 5)                 1285      
                                                                 
=================================================================
Total params: 303,621
Trainable params: 303,621
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 11s 1s/step - loss: 1.5949 - accuracy: 0.2318 - val_loss: 1.9862 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 8s 2s/step - loss: 1.6067 - accuracy: 0.2477 - val_loss: 2.2682 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 6s 1s/step - loss: 1.5367 - accuracy: 0.2500 - val_loss: 2.0308 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 9s 2s/step - loss: 1.5436 - accuracy: 0.2500 - val_loss: 2.1869 - val_accuracy: 0.0000e+00
18/18 [==============================] - 4s 177ms/step
3/3 [==============================] - 1s 165ms/step
18/18 [==============================] - 3s 179ms/step
3/3 [==============================] - 1s 263ms/step
loss

accuracy

val_loss

val_accuracy

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.2	0.228916	0.066667	0.085282

3 times run-

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(3):
  result=LSTM_Model_Tuned_WV(X_train_wvfull_smote, X_test_wvfull, y_train_wvfull_smote, y_test_wvfull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_1895"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1895 (Embedding)  (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_88 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1894 (LSTM)            (None, 256)               295936    
                                                                 
 dense_1894 (Dense)          (None, 5)                 1285      
                                                                 
=================================================================
Total params: 303,621
Trainable params: 303,621
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 10s 1s/step - loss: 1.5903 - accuracy: 0.2227 - val_loss: 2.2046 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 7s 1s/step - loss: 1.5434 - accuracy: 0.2068 - val_loss: 2.0816 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 7s 1s/step - loss: 1.5409 - accuracy: 0.2500 - val_loss: 2.0568 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 7s 1s/step - loss: 1.5352 - accuracy: 0.2500 - val_loss: 2.2663 - val_accuracy: 0.0000e+00
Epoch 5/20
5/5 [==============================] - 7s 1s/step - loss: 1.5256 - accuracy: 0.2500 - val_loss: 2.7630 - val_accuracy: 0.0000e+00
Epoch 6/20
5/5 [==============================] - 7s 1s/step - loss: 1.5196 - accuracy: 0.2500 - val_loss: 2.8388 - val_accuracy: 0.0000e+00
18/18 [==============================] - 4s 179ms/step
3/3 [==============================] - 1s 153ms/step
18/18 [==============================] - 5s 275ms/step
3/3 [==============================] - 1s 160ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_1896"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1896 (Embedding)  (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_89 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1895 (LSTM)            (None, 256)               295936    
                                                                 
 dense_1895 (Dense)          (None, 5)                 1285      
                                                                 
=================================================================
Total params: 303,621
Trainable params: 303,621
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 11s 2s/step - loss: 1.5909 - accuracy: 0.2227 - val_loss: 2.2268 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 6s 1s/step - loss: 1.5409 - accuracy: 0.2159 - val_loss: 2.0982 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 8s 2s/step - loss: 1.5385 - accuracy: 0.2477 - val_loss: 2.0992 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 6s 1s/step - loss: 1.5332 - accuracy: 0.2500 - val_loss: 2.4217 - val_accuracy: 0.0000e+00
Epoch 5/20
5/5 [==============================] - 8s 2s/step - loss: 1.5263 - accuracy: 0.2500 - val_loss: 2.7911 - val_accuracy: 0.0000e+00
18/18 [==============================] - 4s 178ms/step
3/3 [==============================] - 1s 272ms/step
18/18 [==============================] - 4s 242ms/step
3/3 [==============================] - 1s 159ms/step
loss

accuracy

val_loss

val_accuracy

Model: "sequential_1897"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1897 (Embedding)  (None, 219, 32)           6400      
                                                                 
 spatial_dropout1d_90 (Spati  (None, 219, 32)          0         
 alDropout1D)                                                    
                                                                 
 lstm_1896 (LSTM)            (None, 256)               295936    
                                                                 
 dense_1896 (Dense)          (None, 5)                 1285      
                                                                 
=================================================================
Total params: 303,621
Trainable params: 303,621
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 10s 2s/step - loss: 1.5930 - accuracy: 0.2136 - val_loss: 2.0713 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 6s 1s/step - loss: 1.5350 - accuracy: 0.2250 - val_loss: 2.1211 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 8s 2s/step - loss: 1.5332 - accuracy: 0.2568 - val_loss: 2.2398 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 6s 1s/step - loss: 1.5240 - accuracy: 0.2500 - val_loss: 2.7593 - val_accuracy: 0.0000e+00
18/18 [==============================] - 6s 297ms/step
3/3 [==============================] - 1s 153ms/step
18/18 [==============================] - 3s 178ms/step
3/3 [==============================] - 1s 157ms/step
loss

accuracy

val_loss

val_accuracy

Result of all runs:   model  train accuracy  test accuracy  train F1 score  test F1 score
0  LSTM             0.2       0.228916        0.066667       0.085282
1  LSTM             0.2       0.337349        0.066667       0.170194
2  LSTM             0.2       0.337349        0.066667       0.170194

Observations-

1. Results are very much similar to other 2 datasets.
2. Full dataset is giving less accuracy and F1 score in Word2vec.
3. Best achieved accuracy is 33% and F1 score is 17% with tuning methods.

Applying Bidirectional LSTM-

Countvectorizer dataset-

BI_LSTM_Model(X_train_cv, X_test_cv, y_train_cv, y_test_cv)

Model: "sequential_1898"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1898 (Embedding)  (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_91 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 bidirectional (Bidirectiona  (None, 400)              347200    
 l)                                                              
                                                                 
 dense_1897 (Dense)          (None, 5)                 2005      
                                                                 
=================================================================
Total params: 352,405
Trainable params: 352,405
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 33s 2s/step - loss: 1.5350 - accuracy: 0.2748 - val_loss: 1.5433 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 24s 2s/step - loss: 1.5082 - accuracy: 0.3359 - val_loss: 1.5327 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 24s 2s/step - loss: 1.4795 - accuracy: 0.3359 - val_loss: 1.5534 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 25s 2s/step - loss: 1.4728 - accuracy: 0.3206 - val_loss: 1.5459 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 25s 2s/step - loss: 1.4621 - accuracy: 0.3359 - val_loss: 1.5651 - val_accuracy: 0.3333
11/11 [==============================] - 4s 293ms/step
3/3 [==============================] - 1s 160ms/step
11/11 [==============================] - 2s 175ms/step
3/3 [==============================] - 1s 165ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.335366	0.337349	0.168449	0.170194

Countvectorizer Full dataset-

BI_LSTM_Model(X_train_cvfull, X_test_cvfull, y_train_cvfull, y_test_cvfull)

Model: "sequential_1900"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1900 (Embedding)  (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_93 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 bidirectional_2 (Bidirectio  (None, 400)              347200    
 nal)                                                            
                                                                 
 dense_1899 (Dense)          (None, 5)                 2005      
                                                                 
=================================================================
Total params: 352,405
Trainable params: 352,405
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 37s 2s/step - loss: 1.5632 - accuracy: 0.2824 - val_loss: 1.5394 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 27s 2s/step - loss: 1.5112 - accuracy: 0.3359 - val_loss: 1.5400 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 27s 2s/step - loss: 1.4860 - accuracy: 0.3282 - val_loss: 1.5645 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 27s 2s/step - loss: 1.4816 - accuracy: 0.3206 - val_loss: 1.5436 - val_accuracy: 0.3333
11/11 [==============================] - 3s 220ms/step
3/3 [==============================] - 1s 182ms/step
11/11 [==============================] - 2s 192ms/step
3/3 [==============================] - 1s 196ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.335366	0.337349	0.168449	0.170194

TFIDF Dataset-

BI_LSTM_Model(X_train_tfidf, X_test_tfidf, y_train_tfidf, y_test_tfidf)

Model: "sequential_1901"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1901 (Embedding)  (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_94 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 bidirectional_3 (Bidirectio  (None, 400)              347200    
 nal)                                                            
                                                                 
 dense_1900 (Dense)          (None, 5)                 2005      
                                                                 
=================================================================
Total params: 352,405
Trainable params: 352,405
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 32s 2s/step - loss: 1.5649 - accuracy: 0.2710 - val_loss: 1.5396 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 26s 2s/step - loss: 1.5149 - accuracy: 0.3359 - val_loss: 1.5383 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 25s 2s/step - loss: 1.4855 - accuracy: 0.3359 - val_loss: 1.5632 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 25s 2s/step - loss: 1.4845 - accuracy: 0.2977 - val_loss: 1.5434 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 25s 2s/step - loss: 1.4645 - accuracy: 0.3359 - val_loss: 1.5549 - val_accuracy: 0.3333
11/11 [==============================] - 2s 171ms/step
3/3 [==============================] - 1s 164ms/step
11/11 [==============================] - 2s 176ms/step
3/3 [==============================] - 1s 167ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.335366	0.337349	0.168449	0.170194

TFIDF Full dataset-

BI_LSTM_Model(X_train_tfidffull, X_test_tfidffull, y_train_tfidffull, y_test_tfidffull)

Model: "sequential_1902"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1902 (Embedding)  (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_95 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 bidirectional_4 (Bidirectio  (None, 400)              347200    
 nal)                                                            
                                                                 
 dense_1901 (Dense)          (None, 5)                 2005      
                                                                 
=================================================================
Total params: 352,405
Trainable params: 352,405
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 34s 2s/step - loss: 1.5426 - accuracy: 0.2863 - val_loss: 1.5447 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 27s 2s/step - loss: 1.5114 - accuracy: 0.3359 - val_loss: 1.5334 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 27s 2s/step - loss: 1.4787 - accuracy: 0.3359 - val_loss: 1.5462 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 29s 2s/step - loss: 1.4749 - accuracy: 0.3206 - val_loss: 1.5453 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 27s 2s/step - loss: 1.4615 - accuracy: 0.3359 - val_loss: 1.5665 - val_accuracy: 0.3333
11/11 [==============================] - 3s 190ms/step
3/3 [==============================] - 1s 187ms/step
11/11 [==============================] - 2s 193ms/step
3/3 [==============================] - 1s 176ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.335366	0.337349	0.168449	0.170194

Word2vec Dataset-

BI_LSTM_Model(X_train_wv, X_test_wv, y_train_wv, y_test_wv)

Model: "sequential_1903"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1903 (Embedding)  (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d_96 (Spati  (None, 200, 16)          0         
 alDropout1D)                                                    
                                                                 
 bidirectional_5 (Bidirectio  (None, 400)              347200    
 nal)                                                            
                                                                 
 dense_1902 (Dense)          (None, 5)                 2005      
                                                                 
=================================================================
Total params: 352,405
Trainable params: 352,405
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 34s 2s/step - loss: 1.5540 - accuracy: 0.3015 - val_loss: 1.5431 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 25s 2s/step - loss: 1.5173 - accuracy: 0.3359 - val_loss: 1.5409 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 24s 2s/step - loss: 1.4848 - accuracy: 0.3321 - val_loss: 1.5659 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 25s 2s/step - loss: 1.4818 - accuracy: 0.3321 - val_loss: 1.5413 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 25s 2s/step - loss: 1.4660 - accuracy: 0.3359 - val_loss: 1.5554 - val_accuracy: 0.3333
11/11 [==============================] - 3s 215ms/step
3/3 [==============================] - 1s 175ms/step
11/11 [==============================] - 2s 174ms/step
3/3 [==============================] - 1s 159ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.335366	0.337349	0.168449	0.170194

Word2vec Full dataset-

BI_LSTM_Model(X_train_wvfull, X_test_wvfull, y_train_wvfull, y_test_wvfull)

Model: "sequential_1904"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_1904 (Embedding)  (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d_97 (Spati  (None, 219, 16)          0         
 alDropout1D)                                                    
                                                                 
 bidirectional_6 (Bidirectio  (None, 400)              347200    
 nal)                                                            
                                                                 
 dense_1903 (Dense)          (None, 5)                 2005      
                                                                 
=================================================================
Total params: 352,405
Trainable params: 352,405
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/100
14/14 [==============================] - 36s 2s/step - loss: 1.5635 - accuracy: 0.2557 - val_loss: 1.5451 - val_accuracy: 0.3333
Epoch 2/100
14/14 [==============================] - 28s 2s/step - loss: 1.5307 - accuracy: 0.3359 - val_loss: 1.5497 - val_accuracy: 0.3333
Epoch 3/100
14/14 [==============================] - 27s 2s/step - loss: 1.4953 - accuracy: 0.3359 - val_loss: 1.5544 - val_accuracy: 0.3333
Epoch 4/100
14/14 [==============================] - 27s 2s/step - loss: 1.4857 - accuracy: 0.3359 - val_loss: 1.5410 - val_accuracy: 0.3333
Epoch 5/100
14/14 [==============================] - 27s 2s/step - loss: 1.4640 - accuracy: 0.3359 - val_loss: 1.5581 - val_accuracy: 0.3333
Epoch 6/100
14/14 [==============================] - 28s 2s/step - loss: 1.4702 - accuracy: 0.2786 - val_loss: 1.5725 - val_accuracy: 0.3333
Epoch 7/100
14/14 [==============================] - 29s 2s/step - loss: 1.4633 - accuracy: 0.3397 - val_loss: 1.5488 - val_accuracy: 0.3333
11/11 [==============================] - 3s 190ms/step
3/3 [==============================] - 1s 172ms/step
11/11 [==============================] - 3s 236ms/step
3/3 [==============================] - 1s 182ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.335366	0.337349	0.168449	0.170194

Observations-

1. Accuracy is 33% and F1 score is 17% in all datasets.
2. The result in bidirectional LSTM is similar to LSTM model.
3. We did not run tuned method as results are similar to LSTM.

Keras Tuner method to find accuracy for each types of datasets-

Countvectorizer Dataset-

Randomsearch_LSTM(X_train_cv, X_test_cv, y_train_cv, y_test_cv)

Trial 3 Complete [00h 01m 18s]
accuracy: 0.2926829159259796

Best accuracy So Far: 0.3353658616542816
Total elapsed time: 00h 03m 04s
best parameters with Keras tuner method:  {'units': 384}
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding (Embedding)       (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d (SpatialD  (None, 200, 16)          0         
 ropout1D)                                                       
                                                                 
 lstm (LSTM)                 (None, 384)               615936    
                                                                 
 dense (Dense)               (None, 5)                 1925      
                                                                 
=================================================================
Total params: 621,061
Trainable params: 621,061
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/5
11/11 [==============================] - 15s 1s/step - loss: nan - accuracy: 0.3293 - val_loss: nan - val_accuracy: 0.1084
Epoch 2/5
11/11 [==============================] - 13s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084
Epoch 3/5
11/11 [==============================] - 13s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084
Epoch 4/5
11/11 [==============================] - 12s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084
Epoch 5/5
11/11 [==============================] - 12s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084
3/3 [==============================] - 1s 219ms/step

WARNING:tensorflow:Detecting that an object or model or tf.train.Checkpoint is being deleted with unrestored values. See the following logs for the specific values in question. To silence these warnings, use `status.expect_partial()`. See https://www.tensorflow.org/api_docs/python/tf/train/Checkpoint#restorefor details about the status object returned by the restore function.
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.iter
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.beta_1
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.beta_2
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.decay
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.learning_rate

Random Search LSTM accuracy:  0.10843373493975904

TFIDF Dataset-

Randomsearch_LSTM(X_train_tfidf, X_test_tfidf, y_train_tfidf, y_test_tfidf)

Reloading Tuner from ./untitled_project/tuner0.json
Search space summary
Default search space size: 1
units (Int)
{'default': None, 'conditions': [], 'min_value': 32, 'max_value': 512, 'step': 32, 'sampling': 'linear'}
best parameters with Keras tuner method:  {'units': 384}
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding (Embedding)       (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d (SpatialD  (None, 200, 16)          0         
 ropout1D)                                                       
                                                                 
 lstm (LSTM)                 (None, 384)               615936    
                                                                 
 dense (Dense)               (None, 5)                 1925      
                                                                 
=================================================================
Total params: 621,061
Trainable params: 621,061
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/5
11/11 [==============================] - 15s 1s/step - loss: 1.6081 - accuracy: 0.3140 - val_loss: 1.5347 - val_accuracy: 0.3373
Epoch 2/5
11/11 [==============================] - 13s 1s/step - loss: 1.5647 - accuracy: 0.3354 - val_loss: 1.5792 - val_accuracy: 0.3373
Epoch 3/5
11/11 [==============================] - 13s 1s/step - loss: 1.5788 - accuracy: 0.3354 - val_loss: 1.5785 - val_accuracy: 0.3373
Epoch 4/5
11/11 [==============================] - 13s 1s/step - loss: 1.5734 - accuracy: 0.3354 - val_loss: 1.5650 - val_accuracy: 0.3373
Epoch 5/5
11/11 [==============================] - 13s 1s/step - loss: 35950004.0000 - accuracy: 0.3110 - val_loss: 1.5611 - val_accuracy: 0.3373
3/3 [==============================] - 1s 217ms/step

WARNING:tensorflow:Detecting that an object or model or tf.train.Checkpoint is being deleted with unrestored values. See the following logs for the specific values in question. To silence these warnings, use `status.expect_partial()`. See https://www.tensorflow.org/api_docs/python/tf/train/Checkpoint#restorefor details about the status object returned by the restore function.
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.iter
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.beta_1
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.beta_2
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.decay
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.learning_rate

Random Search LSTM accuracy:  0.3373493975903614

Word2vec dataset-

Randomsearch_LSTM(X_train_wv, X_test_wv, y_train_wv, y_test_wv)

Reloading Tuner from ./untitled_project/tuner0.json
Search space summary
Default search space size: 1
units (Int)
{'default': None, 'conditions': [], 'min_value': 32, 'max_value': 512, 'step': 32, 'sampling': 'linear'}
best parameters with Keras tuner method:  {'units': 384}
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding (Embedding)       (None, 200, 16)           3200      
                                                                 
 spatial_dropout1d (SpatialD  (None, 200, 16)          0         
 ropout1D)                                                       
                                                                 
 lstm (LSTM)                 (None, 384)               615936    
                                                                 
 dense (Dense)               (None, 5)                 1925      
                                                                 
=================================================================
Total params: 621,061
Trainable params: 621,061
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/5
11/11 [==============================] - 17s 1s/step - loss: 1.5668 - accuracy: 0.3201 - val_loss: 1.5158 - val_accuracy: 0.3373
Epoch 2/5
11/11 [==============================] - 13s 1s/step - loss: nan - accuracy: 0.3079 - val_loss: nan - val_accuracy: 0.1084
Epoch 3/5
11/11 [==============================] - 13s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084
Epoch 4/5
11/11 [==============================] - 13s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084
Epoch 5/5
11/11 [==============================] - 13s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084
3/3 [==============================] - 1s 219ms/step

WARNING:tensorflow:Detecting that an object or model or tf.train.Checkpoint is being deleted with unrestored values. See the following logs for the specific values in question. To silence these warnings, use `status.expect_partial()`. See https://www.tensorflow.org/api_docs/python/tf/train/Checkpoint#restorefor details about the status object returned by the restore function.
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.iter
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.beta_1
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.beta_2
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.decay
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.learning_rate

Random Search LSTM accuracy:  0.10843373493975904

Overwritting build_keraslstm function to change as per full dataset-

def build_keraslstm(hp):
  in_dim = X_train_cvfull.shape[1]
  tf.random.set_seed(7)
  embedding_vecor_length = 16
  model = Sequential()
  model.add(Embedding(max_features, embedding_vecor_length, input_length=in_dim))
  model.add(SpatialDropout1D(0.2))
  model.add(LSTM(units=hp.Int('units', min_value=32, max_value=512, step=32), activation='relu'))
  model.add(Dense(5 , activation='softmax'))
  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])
  return model

Countvectorizer Full dataset-

Randomsearch_LSTM(X_train_cvfull, X_test_cvfull, y_train_cvfull, y_test_cvfull)

Reloading Tuner from ./untitled_project/tuner0.json
Search space summary
Default search space size: 1
units (Int)
{'default': None, 'conditions': [], 'min_value': 32, 'max_value': 512, 'step': 32, 'sampling': 'linear'}
best parameters with Keras tuner method:  {'units': 384}
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding (Embedding)       (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d (SpatialD  (None, 219, 16)          0         
 ropout1D)                                                       
                                                                 
 lstm (LSTM)                 (None, 384)               615936    
                                                                 
 dense (Dense)               (None, 5)                 1925      
                                                                 
=================================================================
Total params: 621,061
Trainable params: 621,061
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/5
11/11 [==============================] - 24s 2s/step - loss: nan - accuracy: 0.3262 - val_loss: nan - val_accuracy: 0.1084
Epoch 2/5
11/11 [==============================] - 19s 2s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084
Epoch 3/5
11/11 [==============================] - 14s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084
Epoch 4/5
11/11 [==============================] - 14s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084
Epoch 5/5
11/11 [==============================] - 14s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084

WARNING:tensorflow:5 out of the last 38 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7fe2ff57b130> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

3/3 [==============================] - 1s 240ms/step

WARNING:tensorflow:Detecting that an object or model or tf.train.Checkpoint is being deleted with unrestored values. See the following logs for the specific values in question. To silence these warnings, use `status.expect_partial()`. See https://www.tensorflow.org/api_docs/python/tf/train/Checkpoint#restorefor details about the status object returned by the restore function.
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.iter
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.beta_1
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.beta_2
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.decay
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.learning_rate

Random Search LSTM accuracy:  0.10843373493975904

IFIDF full dataset-

Randomsearch_LSTM(X_train_tfidffull, X_test_tfidffull, y_train_tfidffull, y_test_tfidffull)

Reloading Tuner from ./untitled_project/tuner0.json
Search space summary
Default search space size: 1
units (Int)
{'default': None, 'conditions': [], 'min_value': 32, 'max_value': 512, 'step': 32, 'sampling': 'linear'}
best parameters with Keras tuner method:  {'units': 384}
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding (Embedding)       (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d (SpatialD  (None, 219, 16)          0         
 ropout1D)                                                       
                                                                 
 lstm (LSTM)                 (None, 384)               615936    
                                                                 
 dense (Dense)               (None, 5)                 1925      
                                                                 
=================================================================
Total params: 621,061
Trainable params: 621,061
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/5
11/11 [==============================] - 16s 1s/step - loss: 1.5683 - accuracy: 0.3110 - val_loss: 1.4902 - val_accuracy: 0.3373
Epoch 2/5
11/11 [==============================] - 15s 1s/step - loss: nan - accuracy: 0.2409 - val_loss: nan - val_accuracy: 0.1084
Epoch 3/5
11/11 [==============================] - 14s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084
Epoch 4/5
11/11 [==============================] - 14s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084
Epoch 5/5
11/11 [==============================] - 14s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084

WARNING:tensorflow:5 out of the last 13 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7fe2ff57beb0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

3/3 [==============================] - 1s 239ms/step
Random Search LSTM accuracy:  0.10843373493975904

WARNING:tensorflow:Detecting that an object or model or tf.train.Checkpoint is being deleted with unrestored values. See the following logs for the specific values in question. To silence these warnings, use `status.expect_partial()`. See https://www.tensorflow.org/api_docs/python/tf/train/Checkpoint#restorefor details about the status object returned by the restore function.
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.iter
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.beta_1
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.beta_2
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.decay
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.learning_rate

Word2vec Full dataset-

Randomsearch_LSTM(X_train_wvfull, X_test_wvfull, y_train_wvfull, y_test_wvfull)

Reloading Tuner from ./untitled_project/tuner0.json
Search space summary
Default search space size: 1
units (Int)
{'default': None, 'conditions': [], 'min_value': 32, 'max_value': 512, 'step': 32, 'sampling': 'linear'}
best parameters with Keras tuner method:  {'units': 384}
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding (Embedding)       (None, 219, 16)           3200      
                                                                 
 spatial_dropout1d (SpatialD  (None, 219, 16)          0         
 ropout1D)                                                       
                                                                 
 lstm (LSTM)                 (None, 384)               615936    
                                                                 
 dense (Dense)               (None, 5)                 1925      
                                                                 
=================================================================
Total params: 621,061
Trainable params: 621,061
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/5
11/11 [==============================] - 17s 1s/step - loss: 36043326893326336.0000 - accuracy: 0.3171 - val_loss: nan - val_accuracy: 0.1084
Epoch 2/5
11/11 [==============================] - 14s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084
Epoch 3/5
11/11 [==============================] - 14s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084
Epoch 4/5
11/11 [==============================] - 14s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084
Epoch 5/5
11/11 [==============================] - 14s 1s/step - loss: nan - accuracy: 0.1037 - val_loss: nan - val_accuracy: 0.1084
3/3 [==============================] - 1s 237ms/step

WARNING:tensorflow:Detecting that an object or model or tf.train.Checkpoint is being deleted with unrestored values. See the following logs for the specific values in question. To silence these warnings, use `status.expect_partial()`. See https://www.tensorflow.org/api_docs/python/tf/train/Checkpoint#restorefor details about the status object returned by the restore function.
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.iter
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.beta_1
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.beta_2
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.decay
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer.learning_rate

Random Search LSTM accuracy:  0.10843373493975904

Observations-

1. One of Keras tuner method gives accuracy around 33%. Rest results are around 10%.
2. Keras tuner method takes very less time for execution compared to Keras classifier.
3. Keras tuner method can not be considered for a consistent result. We possibly needs to consider Kfold or Stratified Kfold along with both Keras tuner and Keras classifier which will give more accuracy.
4. LSTM in general is giving less accuracy. So a bigger dataset is required to achieve better accuracy.

Final Testing with Glove method on LSTM-

# load the GloVe vectors in a dictionary:

from tqdm import tqdm
import numpy as np

embeddings_index = {}
f = open(r'/content/drive/MyDrive/Colab Notebooks/Projects/NLP/glove.6B.50d.txt',encoding="utf8")

for line in tqdm(f):
    # Splitting the each line
    values = line.split()

    word = values[0]

    coefs = np.array(values[1:], dtype='float32')

    embeddings_index[word] = coefs

f.close()

print('Found %s word vectors.' % len(embeddings_index))

400000it [00:08, 48785.39it/s]

Found 400000 word vectors.

all_embs = np.stack(list(embeddings_index.values()))
emb_mean,emb_std = all_embs.mean(), all_embs.std()
embed_size = all_embs.shape[1]

word_index = tk.word_index

nb_words = min(max_features, len(tk_corpus))
#change below line if computing normal stats is too slow
embedding_matrix = np.random.normal(emb_mean, emb_std, (nb_words, embed_size))
for word, i in word_index.items():
    if i >= max_features: continue
    embedding_vector = embeddings_index.get(word)
    if embedding_vector is not None: embedding_matrix[i] = embedding_vector

def LSTM_Model_Glove (X_train, X_test, y_train, y_test):

  tf.random.set_seed(7)

  model = Sequential()

  model.add(Embedding(nb_words, embed_size,weights=[embedding_matrix], input_length = X_train.shape[1], trainable=True))

  model.add(SpatialDropout1D(0.2))

  model.add(LSTM(200, dropout = 0.2, recurrent_dropout = 0.2))

  model.add(Dense(5, activation = 'softmax'))


  model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

  print(model.summary())

  y_train_cat=to_categorical(y_train)
  y_test_cat=to_categorical(y_test)

  early_stopping = EarlyStopping(monitor='val_loss', mode='min', verbose=0, patience=3)

  history=model.fit(X_train, y_train_cat, validation_split=0.2, epochs = 20, batch_size = 100, callbacks=[early_stopping])

  train_acc = accuracy_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))
  test_acc = accuracy_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1))

  train_f1_score = f1_score(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1), average='weighted')
  test_f1_score = f1_score(np.argmax(y_test_cat, axis=1), np.argmax(model.predict(X_test), axis=1),average='weighted')

  result_kfold_df= pd.DataFrame({'model': ['LSTM'], 'train accuracy': [train_acc], 'test accuracy': [test_acc], 'train F1 score': [train_f1_score], 'test F1 score': [test_f1_score]  })


  # plotting the model architecture
  from tensorflow.keras.utils import plot_model
  plot_model(model, to_file='/content/drive/MyDrive/Colab Notebooks/Projects/NLP/glove_model.png', show_shapes=True, show_layer_names=True)


  # Confusion matrix for the best model
  cm = confusion_matrix(np.argmax(y_train_cat, axis=1), np.argmax(model.predict(X_train), axis=1))

  fig, ax = plt.subplots(nrows = 1, ncols = 1, figsize = (5,4))
  sns.heatmap(cm,
                annot=True,
                fmt='g',
                xticklabels=['I','II','III','IV','V'],
                yticklabels=['I','II','III','IV','V'], ax = ax)
  ax.set_ylabel('Prediction', fontsize = 11)
  ax.set_xlabel('Actual',fontsize = 11)
  ax.set_title(f'Confusion Matrix: LSTM model',fontsize = 14)
  plt.show()

  # print(result_kfold_df)

  # hist= pd.DataFrame(history.history)

  # for col in hist.columns:
  #   print(col)
  #   plt.plot(hist[col])
  #   plt.plot(hist[col])
  #   plt.title('model-'+col)
  #   plt.ylabel(col)
  #   plt.xlabel('epoch')
  #   plt.show()

  # saving the model
  model.save("/content/drive/MyDrive/Colab Notebooks/Projects/NLP/glove_model.h5")

  return result_kfold_df

LSTM_Model_Glove(X_train_tk, X_test_tk, y_train_tk, y_test_tk)

Model: "sequential_6"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_6 (Embedding)     (None, 200, 50)           10000     
                                                                 
 spatial_dropout1d_6 (Spatia  (None, 200, 50)          0         
 lDropout1D)                                                     
                                                                 
 lstm_6 (LSTM)               (None, 200)               200800    
                                                                 
 dense_6 (Dense)             (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 13s 3s/step - loss: 1.5773 - accuracy: 0.2557 - val_loss: 1.5203 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 7s 2s/step - loss: 1.4542 - accuracy: 0.3511 - val_loss: 1.5621 - val_accuracy: 0.2879
Epoch 3/20
3/3 [==============================] - 9s 2s/step - loss: 1.4160 - accuracy: 0.3817 - val_loss: 1.5477 - val_accuracy: 0.2879
Epoch 4/20
3/3 [==============================] - 7s 2s/step - loss: 1.4208 - accuracy: 0.3511 - val_loss: 1.5217 - val_accuracy: 0.3182
11/11 [==============================] - 2s 141ms/step
3/3 [==============================] - 0s 83ms/step
11/11 [==============================] - 1s 91ms/step
3/3 [==============================] - 0s 85ms/step
11/11 [==============================] - 1s 95ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.387195	0.361446	0.333406	0.280572

from PIL import Image
from IPython.display import display

img = Image.open('/content/drive/MyDrive/Colab Notebooks/Projects/NLP/glove_model.png')

display(img)

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(5):
  result=LSTM_Model_Glove(X_train_tk, X_test_tk, y_train_tk, y_test_tk)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_22"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_19 (Embedding)    (None, 200, 50)           10000     
                                                                 
 spatial_dropout1d_19 (Spati  (None, 200, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_19 (LSTM)              (None, 200)               200800    
                                                                 
 dense_19 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 11s 2s/step - loss: 1.5721 - accuracy: 0.2176 - val_loss: 1.5563 - val_accuracy: 0.3182
Epoch 2/20
3/3 [==============================] - 5s 2s/step - loss: 1.4489 - accuracy: 0.3321 - val_loss: 1.5765 - val_accuracy: 0.3030
Epoch 3/20
3/3 [==============================] - 8s 3s/step - loss: 1.4309 - accuracy: 0.3588 - val_loss: 1.5539 - val_accuracy: 0.2727
Epoch 4/20
3/3 [==============================] - 6s 2s/step - loss: 1.4057 - accuracy: 0.3855 - val_loss: 1.5124 - val_accuracy: 0.3182
Epoch 5/20
3/3 [==============================] - 8s 3s/step - loss: 1.4039 - accuracy: 0.3664 - val_loss: 1.4624 - val_accuracy: 0.3333
Epoch 6/20
3/3 [==============================] - 10s 3s/step - loss: 1.3517 - accuracy: 0.4313 - val_loss: 1.4435 - val_accuracy: 0.3333
Epoch 7/20
3/3 [==============================] - 14s 4s/step - loss: 1.3428 - accuracy: 0.4580 - val_loss: 1.4484 - val_accuracy: 0.3030
Epoch 8/20
3/3 [==============================] - 11s 3s/step - loss: 1.3167 - accuracy: 0.4313 - val_loss: 1.4275 - val_accuracy: 0.3485
Epoch 9/20
3/3 [==============================] - 11s 4s/step - loss: 1.3136 - accuracy: 0.4275 - val_loss: 1.4003 - val_accuracy: 0.2879
Epoch 10/20
3/3 [==============================] - 9s 3s/step - loss: 1.2902 - accuracy: 0.4427 - val_loss: 1.4265 - val_accuracy: 0.3333
Epoch 11/20
3/3 [==============================] - 11s 3s/step - loss: 1.2834 - accuracy: 0.4847 - val_loss: 1.3889 - val_accuracy: 0.3636
Epoch 12/20
3/3 [==============================] - 11s 4s/step - loss: 1.2627 - accuracy: 0.4542 - val_loss: 1.3950 - val_accuracy: 0.3788
Epoch 13/20
3/3 [==============================] - 9s 3s/step - loss: 1.2354 - accuracy: 0.4924 - val_loss: 1.3852 - val_accuracy: 0.3788
Epoch 14/20
3/3 [==============================] - 10s 3s/step - loss: 1.2231 - accuracy: 0.4924 - val_loss: 1.3802 - val_accuracy: 0.3636
Epoch 15/20
3/3 [==============================] - 6s 2s/step - loss: 1.2039 - accuracy: 0.5344 - val_loss: 1.3821 - val_accuracy: 0.3485
Epoch 16/20
3/3 [==============================] - 7s 3s/step - loss: 1.1812 - accuracy: 0.5191 - val_loss: 1.4431 - val_accuracy: 0.3333
Epoch 17/20
3/3 [==============================] - 5s 2s/step - loss: 1.1824 - accuracy: 0.5000 - val_loss: 1.4914 - val_accuracy: 0.3182
11/11 [==============================] - 2s 120ms/step
3/3 [==============================] - 0s 124ms/step
11/11 [==============================] - 2s 197ms/step
3/3 [==============================] - 1s 192ms/step
Model: "sequential_23"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_20 (Embedding)    (None, 200, 50)           10000     
                                                                 
 spatial_dropout1d_20 (Spati  (None, 200, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_20 (LSTM)              (None, 200)               200800    
                                                                 
 dense_20 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 9s 2s/step - loss: 1.5571 - accuracy: 0.2634 - val_loss: 1.5609 - val_accuracy: 0.2879
Epoch 2/20
3/3 [==============================] - 8s 3s/step - loss: 1.4534 - accuracy: 0.3397 - val_loss: 1.5750 - val_accuracy: 0.2727
Epoch 3/20
3/3 [==============================] - 5s 2s/step - loss: 1.4300 - accuracy: 0.3397 - val_loss: 1.5312 - val_accuracy: 0.3182
Epoch 4/20
3/3 [==============================] - 7s 3s/step - loss: 1.4000 - accuracy: 0.3779 - val_loss: 1.4797 - val_accuracy: 0.2727
Epoch 5/20
3/3 [==============================] - 6s 2s/step - loss: 1.3876 - accuracy: 0.3702 - val_loss: 1.4279 - val_accuracy: 0.3485
Epoch 6/20
3/3 [==============================] - 6s 2s/step - loss: 1.3375 - accuracy: 0.4198 - val_loss: 1.4242 - val_accuracy: 0.3939
Epoch 7/20
3/3 [==============================] - 7s 2s/step - loss: 1.3346 - accuracy: 0.4466 - val_loss: 1.4041 - val_accuracy: 0.4242
Epoch 8/20
3/3 [==============================] - 5s 2s/step - loss: 1.3277 - accuracy: 0.4122 - val_loss: 1.4034 - val_accuracy: 0.3636
Epoch 9/20
3/3 [==============================] - 7s 2s/step - loss: 1.3083 - accuracy: 0.4389 - val_loss: 1.3984 - val_accuracy: 0.3182
Epoch 10/20
3/3 [==============================] - 5s 2s/step - loss: 1.2868 - accuracy: 0.4695 - val_loss: 1.4026 - val_accuracy: 0.3030
Epoch 11/20
3/3 [==============================] - 8s 3s/step - loss: 1.2669 - accuracy: 0.4809 - val_loss: 1.3886 - val_accuracy: 0.3788
Epoch 12/20
3/3 [==============================] - 5s 2s/step - loss: 1.2667 - accuracy: 0.4389 - val_loss: 1.4165 - val_accuracy: 0.3939
Epoch 13/20
3/3 [==============================] - 6s 2s/step - loss: 1.2465 - accuracy: 0.5305 - val_loss: 1.3658 - val_accuracy: 0.3636
Epoch 14/20
3/3 [==============================] - 7s 2s/step - loss: 1.2231 - accuracy: 0.4542 - val_loss: 1.3463 - val_accuracy: 0.4091
Epoch 15/20
3/3 [==============================] - 5s 2s/step - loss: 1.2140 - accuracy: 0.4885 - val_loss: 1.3866 - val_accuracy: 0.3485
Epoch 16/20
3/3 [==============================] - 7s 3s/step - loss: 1.1937 - accuracy: 0.5382 - val_loss: 1.4120 - val_accuracy: 0.3182
Epoch 17/20
3/3 [==============================] - 5s 2s/step - loss: 1.1892 - accuracy: 0.5076 - val_loss: 1.4149 - val_accuracy: 0.3788
11/11 [==============================] - 2s 114ms/step
3/3 [==============================] - 0s 110ms/step
11/11 [==============================] - 2s 179ms/step
3/3 [==============================] - 1s 183ms/step
Model: "sequential_24"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_21 (Embedding)    (None, 200, 50)           10000     
                                                                 
 spatial_dropout1d_21 (Spati  (None, 200, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_21 (LSTM)              (None, 200)               200800    
                                                                 
 dense_21 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 8s 2s/step - loss: 1.5464 - accuracy: 0.2519 - val_loss: 1.5546 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 7s 3s/step - loss: 1.4473 - accuracy: 0.3473 - val_loss: 1.5776 - val_accuracy: 0.2879
Epoch 3/20
3/3 [==============================] - 7s 3s/step - loss: 1.4221 - accuracy: 0.3511 - val_loss: 1.5541 - val_accuracy: 0.2576
Epoch 4/20
3/3 [==============================] - 7s 3s/step - loss: 1.3933 - accuracy: 0.4008 - val_loss: 1.5083 - val_accuracy: 0.3182
Epoch 5/20
3/3 [==============================] - 5s 2s/step - loss: 1.3858 - accuracy: 0.3740 - val_loss: 1.4631 - val_accuracy: 0.2879
Epoch 6/20
3/3 [==============================] - 7s 2s/step - loss: 1.3408 - accuracy: 0.4656 - val_loss: 1.4585 - val_accuracy: 0.2879
Epoch 7/20
3/3 [==============================] - 6s 2s/step - loss: 1.3287 - accuracy: 0.4618 - val_loss: 1.4386 - val_accuracy: 0.3333
Epoch 8/20
3/3 [==============================] - 5s 2s/step - loss: 1.3155 - accuracy: 0.4237 - val_loss: 1.4262 - val_accuracy: 0.3636
Epoch 9/20
3/3 [==============================] - 7s 2s/step - loss: 1.2935 - accuracy: 0.4351 - val_loss: 1.4006 - val_accuracy: 0.3939
Epoch 10/20
3/3 [==============================] - 5s 2s/step - loss: 1.2753 - accuracy: 0.4809 - val_loss: 1.4184 - val_accuracy: 0.3182
Epoch 11/20
3/3 [==============================] - 7s 3s/step - loss: 1.2635 - accuracy: 0.4924 - val_loss: 1.3820 - val_accuracy: 0.3788
Epoch 12/20
3/3 [==============================] - 5s 2s/step - loss: 1.2426 - accuracy: 0.5000 - val_loss: 1.3824 - val_accuracy: 0.4242
Epoch 13/20
3/3 [==============================] - 6s 2s/step - loss: 1.2211 - accuracy: 0.4809 - val_loss: 1.3841 - val_accuracy: 0.3636
Epoch 14/20
3/3 [==============================] - 6s 2s/step - loss: 1.2020 - accuracy: 0.5191 - val_loss: 1.3608 - val_accuracy: 0.3939
Epoch 15/20
3/3 [==============================] - 5s 2s/step - loss: 1.1870 - accuracy: 0.5458 - val_loss: 1.4101 - val_accuracy: 0.3788
Epoch 16/20
3/3 [==============================] - 7s 3s/step - loss: 1.1819 - accuracy: 0.4962 - val_loss: 1.3883 - val_accuracy: 0.3636
Epoch 17/20
3/3 [==============================] - 5s 2s/step - loss: 1.1795 - accuracy: 0.4924 - val_loss: 1.3770 - val_accuracy: 0.3939
11/11 [==============================] - 2s 137ms/step
3/3 [==============================] - 1s 191ms/step
11/11 [==============================] - 2s 191ms/step
3/3 [==============================] - 1s 192ms/step
Model: "sequential_25"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_22 (Embedding)    (None, 200, 50)           10000     
                                                                 
 spatial_dropout1d_22 (Spati  (None, 200, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_22 (LSTM)              (None, 200)               200800    
                                                                 
 dense_22 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 8s 2s/step - loss: 1.5743 - accuracy: 0.2366 - val_loss: 1.5367 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 7s 3s/step - loss: 1.4422 - accuracy: 0.3282 - val_loss: 1.5469 - val_accuracy: 0.3182
Epoch 3/20
3/3 [==============================] - 5s 2s/step - loss: 1.4333 - accuracy: 0.3321 - val_loss: 1.5257 - val_accuracy: 0.2576
Epoch 4/20
3/3 [==============================] - 7s 3s/step - loss: 1.3969 - accuracy: 0.3931 - val_loss: 1.4756 - val_accuracy: 0.3333
Epoch 5/20
3/3 [==============================] - 5s 2s/step - loss: 1.3897 - accuracy: 0.3779 - val_loss: 1.4078 - val_accuracy: 0.3485
Epoch 6/20
3/3 [==============================] - 9s 3s/step - loss: 1.3356 - accuracy: 0.4198 - val_loss: 1.4252 - val_accuracy: 0.3485
Epoch 7/20
3/3 [==============================] - 5s 2s/step - loss: 1.3477 - accuracy: 0.4237 - val_loss: 1.4023 - val_accuracy: 0.3030
Epoch 8/20
3/3 [==============================] - 5s 2s/step - loss: 1.3186 - accuracy: 0.4427 - val_loss: 1.4067 - val_accuracy: 0.3182
Epoch 9/20
3/3 [==============================] - 7s 2s/step - loss: 1.2929 - accuracy: 0.4504 - val_loss: 1.3875 - val_accuracy: 0.3333
Epoch 10/20
3/3 [==============================] - 5s 2s/step - loss: 1.2840 - accuracy: 0.4504 - val_loss: 1.4043 - val_accuracy: 0.3182
Epoch 11/20
3/3 [==============================] - 7s 3s/step - loss: 1.2478 - accuracy: 0.5000 - val_loss: 1.3759 - val_accuracy: 0.3485
Epoch 12/20
3/3 [==============================] - 5s 2s/step - loss: 1.2447 - accuracy: 0.4656 - val_loss: 1.3853 - val_accuracy: 0.4091
Epoch 13/20
3/3 [==============================] - 6s 2s/step - loss: 1.2367 - accuracy: 0.5305 - val_loss: 1.3558 - val_accuracy: 0.3939
Epoch 14/20
3/3 [==============================] - 7s 2s/step - loss: 1.1906 - accuracy: 0.4924 - val_loss: 1.3596 - val_accuracy: 0.3939
Epoch 15/20
3/3 [==============================] - 5s 2s/step - loss: 1.1931 - accuracy: 0.5000 - val_loss: 1.4100 - val_accuracy: 0.3636
Epoch 16/20
3/3 [==============================] - 7s 3s/step - loss: 1.1962 - accuracy: 0.4885 - val_loss: 1.3896 - val_accuracy: 0.3485
11/11 [==============================] - 2s 180ms/step
3/3 [==============================] - 1s 184ms/step
11/11 [==============================] - 2s 132ms/step
3/3 [==============================] - 0s 112ms/step
Model: "sequential_26"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_23 (Embedding)    (None, 200, 50)           10000     
                                                                 
 spatial_dropout1d_23 (Spati  (None, 200, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_23 (LSTM)              (None, 200)               200800    
                                                                 
 dense_23 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 8s 2s/step - loss: 1.5566 - accuracy: 0.2901 - val_loss: 1.5462 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 7s 2s/step - loss: 1.4626 - accuracy: 0.3130 - val_loss: 1.5770 - val_accuracy: 0.2727
Epoch 3/20
3/3 [==============================] - 5s 2s/step - loss: 1.4268 - accuracy: 0.3550 - val_loss: 1.5458 - val_accuracy: 0.2727
Epoch 4/20
3/3 [==============================] - 7s 3s/step - loss: 1.3922 - accuracy: 0.3779 - val_loss: 1.4905 - val_accuracy: 0.2879
Epoch 5/20
3/3 [==============================] - 5s 2s/step - loss: 1.3817 - accuracy: 0.3740 - val_loss: 1.4369 - val_accuracy: 0.3485
Epoch 6/20
3/3 [==============================] - 6s 2s/step - loss: 1.3216 - accuracy: 0.4580 - val_loss: 1.4243 - val_accuracy: 0.3939
Epoch 7/20
3/3 [==============================] - 6s 2s/step - loss: 1.3418 - accuracy: 0.4504 - val_loss: 1.3855 - val_accuracy: 0.4091
Epoch 8/20
3/3 [==============================] - 5s 2s/step - loss: 1.3077 - accuracy: 0.4351 - val_loss: 1.3919 - val_accuracy: 0.3788
Epoch 9/20
3/3 [==============================] - 9s 3s/step - loss: 1.2865 - accuracy: 0.4809 - val_loss: 1.3813 - val_accuracy: 0.3485
Epoch 10/20
3/3 [==============================] - 5s 2s/step - loss: 1.2675 - accuracy: 0.4237 - val_loss: 1.3990 - val_accuracy: 0.3485
Epoch 11/20
3/3 [==============================] - 6s 2s/step - loss: 1.2508 - accuracy: 0.5153 - val_loss: 1.3682 - val_accuracy: 0.3485
Epoch 12/20
3/3 [==============================] - 6s 2s/step - loss: 1.2502 - accuracy: 0.4504 - val_loss: 1.4157 - val_accuracy: 0.4091
Epoch 13/20
3/3 [==============================] - 5s 2s/step - loss: 1.2405 - accuracy: 0.5305 - val_loss: 1.3348 - val_accuracy: 0.3939
Epoch 14/20
3/3 [==============================] - 7s 2s/step - loss: 1.1981 - accuracy: 0.5267 - val_loss: 1.3269 - val_accuracy: 0.3333
Epoch 15/20
3/3 [==============================] - 5s 2s/step - loss: 1.1850 - accuracy: 0.5267 - val_loss: 1.3599 - val_accuracy: 0.3636
Epoch 16/20
3/3 [==============================] - 7s 3s/step - loss: 1.1377 - accuracy: 0.5344 - val_loss: 1.4117 - val_accuracy: 0.3788
Epoch 17/20
3/3 [==============================] - 5s 2s/step - loss: 1.1787 - accuracy: 0.4847 - val_loss: 1.3606 - val_accuracy: 0.3788
11/11 [==============================] - 2s 115ms/step
3/3 [==============================] - 1s 194ms/step
11/11 [==============================] - 2s 193ms/step
3/3 [==============================] - 0s 110ms/step
Result of all runs:   model  train accuracy  test accuracy  train F1 score  test F1 score
0  LSTM        0.545732       0.277108        0.530909       0.264791
1  LSTM        0.506098       0.301205        0.481058       0.248341
2  LSTM        0.530488       0.337349        0.508256       0.290590
3  LSTM        0.533537       0.301205        0.515501       0.293400
4  LSTM        0.506098       0.277108        0.491809       0.275645

LSTM_Model_Glove(X_train_tkfull, X_test_tkfull, y_train_tkfull, y_test_tkfull)

Model: "sequential_27"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_24 (Embedding)    (None, 219, 50)           10000     
                                                                 
 spatial_dropout1d_24 (Spati  (None, 219, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_24 (LSTM)              (None, 200)               200800    
                                                                 
 dense_24 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 10s 2s/step - loss: 1.5419 - accuracy: 0.2824 - val_loss: 1.5356 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 6s 2s/step - loss: 1.4535 - accuracy: 0.3321 - val_loss: 1.5408 - val_accuracy: 0.3030
Epoch 3/20
3/3 [==============================] - 5s 2s/step - loss: 1.4294 - accuracy: 0.3435 - val_loss: 1.5102 - val_accuracy: 0.3182
Epoch 4/20
3/3 [==============================] - 8s 2s/step - loss: 1.3883 - accuracy: 0.3969 - val_loss: 1.4755 - val_accuracy: 0.3485
Epoch 5/20
3/3 [==============================] - 6s 2s/step - loss: 1.3725 - accuracy: 0.3626 - val_loss: 1.4435 - val_accuracy: 0.3333
Epoch 6/20
3/3 [==============================] - 8s 3s/step - loss: 1.3291 - accuracy: 0.4122 - val_loss: 1.4696 - val_accuracy: 0.3333
Epoch 7/20
3/3 [==============================] - 5s 2s/step - loss: 1.3328 - accuracy: 0.4313 - val_loss: 1.4433 - val_accuracy: 0.3030
Epoch 8/20
3/3 [==============================] - 8s 3s/step - loss: 1.3218 - accuracy: 0.4237 - val_loss: 1.4359 - val_accuracy: 0.3485
Epoch 9/20
3/3 [==============================] - 5s 2s/step - loss: 1.2779 - accuracy: 0.4504 - val_loss: 1.4186 - val_accuracy: 0.3333
Epoch 10/20
3/3 [==============================] - 8s 3s/step - loss: 1.2806 - accuracy: 0.4427 - val_loss: 1.4475 - val_accuracy: 0.2879
Epoch 11/20
3/3 [==============================] - 6s 2s/step - loss: 1.2589 - accuracy: 0.4924 - val_loss: 1.3890 - val_accuracy: 0.3333
Epoch 12/20
3/3 [==============================] - 7s 2s/step - loss: 1.2635 - accuracy: 0.4580 - val_loss: 1.4277 - val_accuracy: 0.3333
Epoch 13/20
3/3 [==============================] - 6s 2s/step - loss: 1.2352 - accuracy: 0.4771 - val_loss: 1.3997 - val_accuracy: 0.3636
Epoch 14/20
3/3 [==============================] - 6s 2s/step - loss: 1.1878 - accuracy: 0.5458 - val_loss: 1.3702 - val_accuracy: 0.3333
Epoch 15/20
3/3 [==============================] - 7s 2s/step - loss: 1.1605 - accuracy: 0.5267 - val_loss: 1.4447 - val_accuracy: 0.3788
Epoch 16/20
3/3 [==============================] - 5s 2s/step - loss: 1.1661 - accuracy: 0.5153 - val_loss: 1.4514 - val_accuracy: 0.2879
Epoch 17/20
3/3 [==============================] - 8s 2s/step - loss: 1.1696 - accuracy: 0.4924 - val_loss: 1.4339 - val_accuracy: 0.3939
11/11 [==============================] - 2s 125ms/step
3/3 [==============================] - 0s 116ms/step
11/11 [==============================] - 1s 124ms/step
3/3 [==============================] - 0s 117ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.518293	0.301205	0.499856	0.29041

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(5):
  result=LSTM_Model_Glove(X_train_tkfull, X_test_tkfull, y_train_tkfull, y_test_tkfull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_28"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_25 (Embedding)    (None, 219, 50)           10000     
                                                                 
 spatial_dropout1d_25 (Spati  (None, 219, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_25 (LSTM)              (None, 200)               200800    
                                                                 
 dense_25 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 11s 2s/step - loss: 1.5517 - accuracy: 0.2863 - val_loss: 1.5528 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 8s 3s/step - loss: 1.4419 - accuracy: 0.3511 - val_loss: 1.5509 - val_accuracy: 0.2879
Epoch 3/20
3/3 [==============================] - 8s 3s/step - loss: 1.4199 - accuracy: 0.3397 - val_loss: 1.5055 - val_accuracy: 0.3333
Epoch 4/20
3/3 [==============================] - 8s 3s/step - loss: 1.3721 - accuracy: 0.4160 - val_loss: 1.4380 - val_accuracy: 0.3636
Epoch 5/20
3/3 [==============================] - 5s 2s/step - loss: 1.3522 - accuracy: 0.3931 - val_loss: 1.3942 - val_accuracy: 0.4394
Epoch 6/20
3/3 [==============================] - 8s 3s/step - loss: 1.3146 - accuracy: 0.4275 - val_loss: 1.3976 - val_accuracy: 0.4091
Epoch 7/20
3/3 [==============================] - 5s 2s/step - loss: 1.3130 - accuracy: 0.4427 - val_loss: 1.4036 - val_accuracy: 0.3788
Epoch 8/20
3/3 [==============================] - 7s 3s/step - loss: 1.3159 - accuracy: 0.4389 - val_loss: 1.4510 - val_accuracy: 0.3636
11/11 [==============================] - 2s 129ms/step
3/3 [==============================] - 0s 126ms/step
11/11 [==============================] - 1s 126ms/step
3/3 [==============================] - 0s 121ms/step
Model: "sequential_29"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_26 (Embedding)    (None, 219, 50)           10000     
                                                                 
 spatial_dropout1d_26 (Spati  (None, 219, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_26 (LSTM)              (None, 200)               200800    
                                                                 
 dense_26 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 10s 2s/step - loss: 1.5693 - accuracy: 0.2443 - val_loss: 1.5670 - val_accuracy: 0.3182
Epoch 2/20
3/3 [==============================] - 6s 2s/step - loss: 1.4519 - accuracy: 0.3244 - val_loss: 1.5966 - val_accuracy: 0.2727
Epoch 3/20
3/3 [==============================] - 7s 2s/step - loss: 1.4313 - accuracy: 0.3473 - val_loss: 1.5464 - val_accuracy: 0.3182
Epoch 4/20
3/3 [==============================] - 6s 2s/step - loss: 1.3892 - accuracy: 0.3779 - val_loss: 1.4992 - val_accuracy: 0.3485
Epoch 5/20
3/3 [==============================] - 8s 2s/step - loss: 1.3823 - accuracy: 0.3969 - val_loss: 1.4534 - val_accuracy: 0.3788
Epoch 6/20
3/3 [==============================] - 5s 2s/step - loss: 1.3344 - accuracy: 0.4237 - val_loss: 1.4405 - val_accuracy: 0.3939
Epoch 7/20
3/3 [==============================] - 8s 3s/step - loss: 1.3375 - accuracy: 0.4351 - val_loss: 1.4358 - val_accuracy: 0.3788
Epoch 8/20
3/3 [==============================] - 6s 2s/step - loss: 1.3255 - accuracy: 0.4313 - val_loss: 1.4247 - val_accuracy: 0.3788
Epoch 9/20
3/3 [==============================] - 8s 3s/step - loss: 1.2859 - accuracy: 0.4656 - val_loss: 1.4043 - val_accuracy: 0.3485
Epoch 10/20
3/3 [==============================] - 6s 2s/step - loss: 1.2837 - accuracy: 0.4427 - val_loss: 1.4398 - val_accuracy: 0.3333
Epoch 11/20
3/3 [==============================] - 11s 4s/step - loss: 1.2650 - accuracy: 0.4962 - val_loss: 1.3903 - val_accuracy: 0.3788
Epoch 12/20
3/3 [==============================] - 12s 4s/step - loss: 1.2619 - accuracy: 0.4695 - val_loss: 1.4045 - val_accuracy: 0.3939
Epoch 13/20
3/3 [==============================] - 12s 4s/step - loss: 1.2361 - accuracy: 0.5153 - val_loss: 1.3838 - val_accuracy: 0.3485
Epoch 14/20
3/3 [==============================] - 10s 3s/step - loss: 1.1869 - accuracy: 0.5305 - val_loss: 1.3678 - val_accuracy: 0.3788
Epoch 15/20
3/3 [==============================] - 12s 5s/step - loss: 1.1770 - accuracy: 0.5382 - val_loss: 1.3885 - val_accuracy: 0.3939
Epoch 16/20
3/3 [==============================] - 11s 4s/step - loss: 1.1602 - accuracy: 0.5534 - val_loss: 1.4455 - val_accuracy: 0.3636
Epoch 17/20
3/3 [==============================] - 11s 4s/step - loss: 1.1460 - accuracy: 0.5382 - val_loss: 1.4158 - val_accuracy: 0.3333
11/11 [==============================] - 3s 222ms/step
3/3 [==============================] - 1s 199ms/step
11/11 [==============================] - 2s 225ms/step
3/3 [==============================] - 1s 212ms/step
Model: "sequential_30"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_27 (Embedding)    (None, 219, 50)           10000     
                                                                 
 spatial_dropout1d_27 (Spati  (None, 219, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_27 (LSTM)              (None, 200)               200800    
                                                                 
 dense_27 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 19s 4s/step - loss: 1.5443 - accuracy: 0.2710 - val_loss: 1.5775 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 10s 3s/step - loss: 1.4670 - accuracy: 0.3206 - val_loss: 1.5620 - val_accuracy: 0.2727
Epoch 3/20
3/3 [==============================] - 11s 4s/step - loss: 1.4294 - accuracy: 0.3511 - val_loss: 1.5204 - val_accuracy: 0.2727
Epoch 4/20
3/3 [==============================] - 13s 5s/step - loss: 1.3918 - accuracy: 0.3817 - val_loss: 1.4987 - val_accuracy: 0.3182
Epoch 5/20
3/3 [==============================] - 9s 3s/step - loss: 1.3956 - accuracy: 0.3740 - val_loss: 1.4683 - val_accuracy: 0.3182
Epoch 6/20
3/3 [==============================] - 11s 3s/step - loss: 1.3433 - accuracy: 0.4122 - val_loss: 1.4700 - val_accuracy: 0.2879
Epoch 7/20
3/3 [==============================] - 12s 4s/step - loss: 1.3321 - accuracy: 0.4313 - val_loss: 1.4810 - val_accuracy: 0.3333
Epoch 8/20
3/3 [==============================] - 10s 3s/step - loss: 1.3237 - accuracy: 0.4466 - val_loss: 1.4490 - val_accuracy: 0.3636
Epoch 9/20
3/3 [==============================] - 10s 3s/step - loss: 1.2999 - accuracy: 0.4351 - val_loss: 1.3912 - val_accuracy: 0.3182
Epoch 10/20
3/3 [==============================] - 8s 3s/step - loss: 1.2891 - accuracy: 0.4466 - val_loss: 1.4095 - val_accuracy: 0.3485
Epoch 11/20
3/3 [==============================] - 5s 2s/step - loss: 1.2543 - accuracy: 0.5000 - val_loss: 1.3688 - val_accuracy: 0.3485
Epoch 12/20
3/3 [==============================] - 7s 2s/step - loss: 1.2700 - accuracy: 0.4466 - val_loss: 1.4336 - val_accuracy: 0.3636
Epoch 13/20
3/3 [==============================] - 6s 2s/step - loss: 1.2439 - accuracy: 0.4847 - val_loss: 1.3955 - val_accuracy: 0.3485
Epoch 14/20
3/3 [==============================] - 6s 2s/step - loss: 1.2019 - accuracy: 0.5115 - val_loss: 1.3764 - val_accuracy: 0.3485
11/11 [==============================] - 3s 211ms/step
3/3 [==============================] - 0s 119ms/step
11/11 [==============================] - 1s 127ms/step
3/3 [==============================] - 0s 124ms/step
Model: "sequential_31"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_28 (Embedding)    (None, 219, 50)           10000     
                                                                 
 spatial_dropout1d_28 (Spati  (None, 219, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_28 (LSTM)              (None, 200)               200800    
                                                                 
 dense_28 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 11s 3s/step - loss: 1.5777 - accuracy: 0.2366 - val_loss: 1.5586 - val_accuracy: 0.3485
Epoch 2/20
3/3 [==============================] - 5s 2s/step - loss: 1.4562 - accuracy: 0.3359 - val_loss: 1.5700 - val_accuracy: 0.3636
Epoch 3/20
3/3 [==============================] - 7s 2s/step - loss: 1.4251 - accuracy: 0.3435 - val_loss: 1.5323 - val_accuracy: 0.3485
Epoch 4/20
3/3 [==============================] - 6s 2s/step - loss: 1.3958 - accuracy: 0.3931 - val_loss: 1.4853 - val_accuracy: 0.3030
Epoch 5/20
3/3 [==============================] - 6s 2s/step - loss: 1.3965 - accuracy: 0.3817 - val_loss: 1.4430 - val_accuracy: 0.3485
Epoch 6/20
3/3 [==============================] - 8s 2s/step - loss: 1.3386 - accuracy: 0.4084 - val_loss: 1.4406 - val_accuracy: 0.3485
Epoch 7/20
3/3 [==============================] - 5s 2s/step - loss: 1.3315 - accuracy: 0.4389 - val_loss: 1.4730 - val_accuracy: 0.3182
Epoch 8/20
3/3 [==============================] - 10s 4s/step - loss: 1.3420 - accuracy: 0.4198 - val_loss: 1.4284 - val_accuracy: 0.3485
Epoch 9/20
3/3 [==============================] - 6s 2s/step - loss: 1.2940 - accuracy: 0.4046 - val_loss: 1.3936 - val_accuracy: 0.3333
Epoch 10/20
3/3 [==============================] - 8s 2s/step - loss: 1.2880 - accuracy: 0.4618 - val_loss: 1.4391 - val_accuracy: 0.3030
Epoch 11/20
3/3 [==============================] - 6s 2s/step - loss: 1.2866 - accuracy: 0.4542 - val_loss: 1.3969 - val_accuracy: 0.3788
Epoch 12/20
3/3 [==============================] - 8s 3s/step - loss: 1.2677 - accuracy: 0.4580 - val_loss: 1.3840 - val_accuracy: 0.3636
Epoch 13/20
3/3 [==============================] - 6s 2s/step - loss: 1.2380 - accuracy: 0.4885 - val_loss: 1.3888 - val_accuracy: 0.3939
Epoch 14/20
3/3 [==============================] - 8s 3s/step - loss: 1.2114 - accuracy: 0.5534 - val_loss: 1.4071 - val_accuracy: 0.3939
Epoch 15/20
3/3 [==============================] - 6s 2s/step - loss: 1.1777 - accuracy: 0.5573 - val_loss: 1.3786 - val_accuracy: 0.3788
Epoch 16/20
3/3 [==============================] - 8s 3s/step - loss: 1.1924 - accuracy: 0.5382 - val_loss: 1.4067 - val_accuracy: 0.3939
Epoch 17/20
3/3 [==============================] - 6s 2s/step - loss: 1.1732 - accuracy: 0.5000 - val_loss: 1.3868 - val_accuracy: 0.4242
Epoch 18/20
3/3 [==============================] - 6s 2s/step - loss: 1.1648 - accuracy: 0.5153 - val_loss: 1.3933 - val_accuracy: 0.3485
11/11 [==============================] - 3s 196ms/step
3/3 [==============================] - 0s 116ms/step
11/11 [==============================] - 1s 126ms/step
3/3 [==============================] - 0s 124ms/step
Model: "sequential_32"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_29 (Embedding)    (None, 219, 50)           10000     
                                                                 
 spatial_dropout1d_29 (Spati  (None, 219, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_29 (LSTM)              (None, 200)               200800    
                                                                 
 dense_29 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
3/3 [==============================] - 11s 3s/step - loss: 1.5560 - accuracy: 0.2824 - val_loss: 1.5306 - val_accuracy: 0.3333
Epoch 2/20
3/3 [==============================] - 6s 2s/step - loss: 1.4612 - accuracy: 0.3244 - val_loss: 1.5667 - val_accuracy: 0.3030
Epoch 3/20
3/3 [==============================] - 7s 3s/step - loss: 1.4443 - accuracy: 0.3282 - val_loss: 1.5472 - val_accuracy: 0.3030
Epoch 4/20
3/3 [==============================] - 6s 2s/step - loss: 1.3974 - accuracy: 0.3893 - val_loss: 1.4919 - val_accuracy: 0.3030
Epoch 5/20
3/3 [==============================] - 6s 2s/step - loss: 1.3832 - accuracy: 0.3931 - val_loss: 1.4479 - val_accuracy: 0.3485
Epoch 6/20
3/3 [==============================] - 7s 2s/step - loss: 1.3251 - accuracy: 0.4122 - val_loss: 1.4326 - val_accuracy: 0.3636
Epoch 7/20
3/3 [==============================] - 6s 2s/step - loss: 1.3357 - accuracy: 0.4313 - val_loss: 1.4381 - val_accuracy: 0.3636
Epoch 8/20
3/3 [==============================] - 8s 2s/step - loss: 1.3172 - accuracy: 0.4237 - val_loss: 1.4152 - val_accuracy: 0.3485
Epoch 9/20
3/3 [==============================] - 6s 2s/step - loss: 1.2938 - accuracy: 0.4313 - val_loss: 1.3893 - val_accuracy: 0.3485
Epoch 10/20
3/3 [==============================] - 8s 3s/step - loss: 1.2860 - accuracy: 0.4351 - val_loss: 1.4252 - val_accuracy: 0.3788
Epoch 11/20
3/3 [==============================] - 6s 2s/step - loss: 1.2711 - accuracy: 0.4618 - val_loss: 1.3818 - val_accuracy: 0.3636
Epoch 12/20
3/3 [==============================] - 8s 3s/step - loss: 1.2448 - accuracy: 0.5076 - val_loss: 1.3965 - val_accuracy: 0.3788
Epoch 13/20
3/3 [==============================] - 6s 2s/step - loss: 1.2357 - accuracy: 0.5229 - val_loss: 1.3971 - val_accuracy: 0.4091
Epoch 14/20
3/3 [==============================] - 10s 4s/step - loss: 1.1977 - accuracy: 0.5534 - val_loss: 1.3637 - val_accuracy: 0.4091
Epoch 15/20
3/3 [==============================] - 6s 2s/step - loss: 1.1853 - accuracy: 0.5038 - val_loss: 1.3855 - val_accuracy: 0.3485
Epoch 16/20
3/3 [==============================] - 7s 3s/step - loss: 1.1680 - accuracy: 0.5534 - val_loss: 1.4709 - val_accuracy: 0.3636
Epoch 17/20
3/3 [==============================] - 6s 2s/step - loss: 1.1682 - accuracy: 0.5000 - val_loss: 1.3874 - val_accuracy: 0.4242
11/11 [==============================] - 2s 131ms/step
3/3 [==============================] - 1s 200ms/step
11/11 [==============================] - 2s 129ms/step
3/3 [==============================] - 0s 133ms/step
Result of all runs:   model  train accuracy  test accuracy  train F1 score  test F1 score
0  LSTM        0.536585       0.325301        0.519533       0.308342
1  LSTM        0.539634       0.289157        0.520822       0.244164
2  LSTM        0.521341       0.325301        0.494944       0.285159
3  LSTM        0.515244       0.289157        0.494821       0.256884
4  LSTM        0.469512       0.337349        0.438782       0.293453

LSTM_Model_Glove(X_train_tkfull_smote, X_test_tkfull, y_train_tkfull_smote, y_test_tkfull)

Model: "sequential_35"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_32 (Embedding)    (None, 219, 50)           10000     
                                                                 
 spatial_dropout1d_32 (Spati  (None, 219, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_32 (LSTM)              (None, 200)               200800    
                                                                 
 dense_32 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 14s 2s/step - loss: 1.5588 - accuracy: 0.2136 - val_loss: 2.9737 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 11s 2s/step - loss: 1.5093 - accuracy: 0.3114 - val_loss: 3.2763 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 11s 2s/step - loss: 1.4801 - accuracy: 0.2864 - val_loss: 2.7975 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 11s 2s/step - loss: 1.4240 - accuracy: 0.3909 - val_loss: 2.5443 - val_accuracy: 0.0000e+00
Epoch 5/20
5/5 [==============================] - 10s 2s/step - loss: 1.3987 - accuracy: 0.4114 - val_loss: 2.6332 - val_accuracy: 0.0000e+00
Epoch 6/20
5/5 [==============================] - 11s 2s/step - loss: 1.3538 - accuracy: 0.4341 - val_loss: 3.0187 - val_accuracy: 0.0091
Epoch 7/20
5/5 [==============================] - 11s 2s/step - loss: 1.3332 - accuracy: 0.4273 - val_loss: 2.9929 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 156ms/step
3/3 [==============================] - 0s 124ms/step
18/18 [==============================] - 2s 128ms/step
3/3 [==============================] - 0s 120ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.378182	0.313253	0.317823	0.291587

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(5):
  result=LSTM_Model_Glove(X_train_tkfull_smote, X_test_tkfull, y_train_tkfull_smote, y_test_tkfull)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_42"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_39 (Embedding)    (None, 219, 50)           10000     
                                                                 
 spatial_dropout1d_39 (Spati  (None, 219, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_39 (LSTM)              (None, 200)               200800    
                                                                 
 dense_39 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 15s 2s/step - loss: 1.5591 - accuracy: 0.2477 - val_loss: 3.0126 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 11s 2s/step - loss: 1.4969 - accuracy: 0.3318 - val_loss: 3.2637 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 11s 2s/step - loss: 1.4748 - accuracy: 0.3159 - val_loss: 2.8156 - val_accuracy: 0.0091
Epoch 4/20
5/5 [==============================] - 11s 2s/step - loss: 1.4037 - accuracy: 0.3909 - val_loss: 2.6841 - val_accuracy: 0.0000e+00
Epoch 5/20
5/5 [==============================] - 11s 2s/step - loss: 1.3710 - accuracy: 0.4159 - val_loss: 2.5536 - val_accuracy: 0.0000e+00
Epoch 6/20
5/5 [==============================] - 11s 2s/step - loss: 1.3458 - accuracy: 0.4432 - val_loss: 3.0668 - val_accuracy: 0.0091
Epoch 7/20
5/5 [==============================] - 11s 2s/step - loss: 1.3102 - accuracy: 0.4364 - val_loss: 3.1735 - val_accuracy: 0.0000e+00
Epoch 8/20
5/5 [==============================] - 10s 2s/step - loss: 1.2899 - accuracy: 0.4477 - val_loss: 3.3671 - val_accuracy: 0.0091
18/18 [==============================] - 4s 181ms/step
3/3 [==============================] - 0s 114ms/step
18/18 [==============================] - 2s 127ms/step
3/3 [==============================] - 0s 115ms/step
Model: "sequential_43"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_40 (Embedding)    (None, 219, 50)           10000     
                                                                 
 spatial_dropout1d_40 (Spati  (None, 219, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_40 (LSTM)              (None, 200)               200800    
                                                                 
 dense_40 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 15s 2s/step - loss: 1.5571 - accuracy: 0.2455 - val_loss: 2.9311 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 11s 2s/step - loss: 1.4943 - accuracy: 0.3045 - val_loss: 3.1419 - val_accuracy: 0.0091
Epoch 3/20
5/5 [==============================] - 11s 2s/step - loss: 1.4522 - accuracy: 0.3591 - val_loss: 2.8056 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 9s 2s/step - loss: 1.3968 - accuracy: 0.4114 - val_loss: 2.6518 - val_accuracy: 0.0000e+00
Epoch 5/20
5/5 [==============================] - 11s 2s/step - loss: 1.3670 - accuracy: 0.4182 - val_loss: 2.7389 - val_accuracy: 0.0000e+00
Epoch 6/20
5/5 [==============================] - 12s 2s/step - loss: 1.3306 - accuracy: 0.4432 - val_loss: 3.3599 - val_accuracy: 0.0000e+00
Epoch 7/20
5/5 [==============================] - 11s 2s/step - loss: 1.3125 - accuracy: 0.4477 - val_loss: 3.3759 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 127ms/step
3/3 [==============================] - 0s 127ms/step
18/18 [==============================] - 3s 156ms/step
3/3 [==============================] - 1s 196ms/step
Model: "sequential_44"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_41 (Embedding)    (None, 219, 50)           10000     
                                                                 
 spatial_dropout1d_41 (Spati  (None, 219, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_41 (LSTM)              (None, 200)               200800    
                                                                 
 dense_41 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 17s 3s/step - loss: 1.5412 - accuracy: 0.2682 - val_loss: 3.1338 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 9s 2s/step - loss: 1.5069 - accuracy: 0.3136 - val_loss: 3.3001 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 11s 2s/step - loss: 1.4452 - accuracy: 0.3750 - val_loss: 2.8361 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 11s 2s/step - loss: 1.4046 - accuracy: 0.4000 - val_loss: 2.5261 - val_accuracy: 0.0000e+00
Epoch 5/20
5/5 [==============================] - 11s 2s/step - loss: 1.3722 - accuracy: 0.4250 - val_loss: 2.7278 - val_accuracy: 0.0000e+00
Epoch 6/20
5/5 [==============================] - 10s 2s/step - loss: 1.3444 - accuracy: 0.4659 - val_loss: 3.2778 - val_accuracy: 0.0000e+00
Epoch 7/20
5/5 [==============================] - 11s 2s/step - loss: 1.3218 - accuracy: 0.4227 - val_loss: 3.2344 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 132ms/step
3/3 [==============================] - 1s 198ms/step
18/18 [==============================] - 3s 168ms/step
3/3 [==============================] - 0s 128ms/step
Model: "sequential_45"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_42 (Embedding)    (None, 219, 50)           10000     
                                                                 
 spatial_dropout1d_42 (Spati  (None, 219, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_42 (LSTM)              (None, 200)               200800    
                                                                 
 dense_42 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 15s 3s/step - loss: 1.5576 - accuracy: 0.2432 - val_loss: 2.9611 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 12s 2s/step - loss: 1.5003 - accuracy: 0.3250 - val_loss: 3.3302 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 12s 2s/step - loss: 1.4619 - accuracy: 0.3455 - val_loss: 2.8829 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 9s 2s/step - loss: 1.4001 - accuracy: 0.4114 - val_loss: 2.5763 - val_accuracy: 0.0000e+00
Epoch 5/20
5/5 [==============================] - 11s 2s/step - loss: 1.3720 - accuracy: 0.4227 - val_loss: 2.5806 - val_accuracy: 0.0000e+00
Epoch 6/20
5/5 [==============================] - 11s 2s/step - loss: 1.3437 - accuracy: 0.4341 - val_loss: 3.0974 - val_accuracy: 0.0091
Epoch 7/20
5/5 [==============================] - 14s 3s/step - loss: 1.3279 - accuracy: 0.4341 - val_loss: 3.0714 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 127ms/step
3/3 [==============================] - 0s 122ms/step
18/18 [==============================] - 2s 129ms/step
3/3 [==============================] - 0s 119ms/step
Model: "sequential_46"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_43 (Embedding)    (None, 219, 50)           10000     
                                                                 
 spatial_dropout1d_43 (Spati  (None, 219, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_43 (LSTM)              (None, 200)               200800    
                                                                 
 dense_43 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 16s 2s/step - loss: 1.5534 - accuracy: 0.2545 - val_loss: 3.1087 - val_accuracy: 0.0091
Epoch 2/20
5/5 [==============================] - 10s 2s/step - loss: 1.5005 - accuracy: 0.3318 - val_loss: 3.2953 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 11s 2s/step - loss: 1.4657 - accuracy: 0.3205 - val_loss: 2.7825 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 12s 2s/step - loss: 1.4088 - accuracy: 0.4045 - val_loss: 2.5247 - val_accuracy: 0.0000e+00
Epoch 5/20
5/5 [==============================] - 12s 2s/step - loss: 1.3866 - accuracy: 0.4159 - val_loss: 2.6052 - val_accuracy: 0.0000e+00
Epoch 6/20
5/5 [==============================] - 9s 2s/step - loss: 1.3388 - accuracy: 0.4545 - val_loss: 3.0265 - val_accuracy: 0.0091
Epoch 7/20
5/5 [==============================] - 11s 2s/step - loss: 1.3141 - accuracy: 0.4227 - val_loss: 3.1401 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 130ms/step
3/3 [==============================] - 0s 120ms/step
18/18 [==============================] - 2s 129ms/step
3/3 [==============================] - 0s 130ms/step
Result of all runs:   model  train accuracy  test accuracy  train F1 score  test F1 score
0  LSTM        0.394545       0.277108        0.326318       0.256018
1  LSTM        0.390909       0.313253        0.336673       0.297220
2  LSTM        0.394545       0.313253        0.330349       0.284560
3  LSTM        0.389091       0.313253        0.333106       0.299287
4  LSTM        0.394545       0.301205        0.337728       0.282149

LSTM_Model_Glove(X_train_tk_smote, X_test_tk, y_train_tk_smote, y_test_tk)

Model: "sequential_36"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_33 (Embedding)    (None, 200, 50)           10000     
                                                                 
 spatial_dropout1d_33 (Spati  (None, 200, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_33 (LSTM)              (None, 200)               200800    
                                                                 
 dense_33 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 20s 3s/step - loss: 1.5746 - accuracy: 0.2114 - val_loss: 2.9547 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 19s 4s/step - loss: 1.5219 - accuracy: 0.2909 - val_loss: 3.3709 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 16s 3s/step - loss: 1.4694 - accuracy: 0.3523 - val_loss: 2.8875 - val_accuracy: 0.0091
Epoch 4/20
5/5 [==============================] - 16s 3s/step - loss: 1.4220 - accuracy: 0.3932 - val_loss: 2.6233 - val_accuracy: 0.0000e+00
Epoch 5/20
5/5 [==============================] - 15s 3s/step - loss: 1.3799 - accuracy: 0.4250 - val_loss: 2.5768 - val_accuracy: 0.0000e+00
Epoch 6/20
5/5 [==============================] - 9s 2s/step - loss: 1.3555 - accuracy: 0.4318 - val_loss: 2.9579 - val_accuracy: 0.0000e+00
Epoch 7/20
5/5 [==============================] - 10s 2s/step - loss: 1.3286 - accuracy: 0.4318 - val_loss: 2.9906 - val_accuracy: 0.0091
Epoch 8/20
5/5 [==============================] - 10s 2s/step - loss: 1.2849 - accuracy: 0.4364 - val_loss: 3.4212 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 117ms/step
3/3 [==============================] - 0s 112ms/step
18/18 [==============================] - 2s 118ms/step
3/3 [==============================] - 0s 107ms/step

	model	train accuracy	test accuracy	train F1 score	test F1 score
0	LSTM	0.398182	0.337349	0.32392	0.303486

result_df = pd.DataFrame()
result= pd.DataFrame()

for i in range(5):
  result=LSTM_Model_Glove(X_train_tk_smote, X_test_tk, y_train_tk_smote, y_test_tk)
  result_df= pd.concat([result,result_df]).reset_index(drop=True)


print ('Result of all runs:', result_df)

plt.plot(result_df['train F1 score'])
plt.plot(result_df['test F1 score'])
plt.title('Model F1 score')
plt.ylabel('F1 score')
plt.xlabel('times')
plt.show()

Model: "sequential_37"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_34 (Embedding)    (None, 200, 50)           10000     
                                                                 
 spatial_dropout1d_34 (Spati  (None, 200, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_34 (LSTM)              (None, 200)               200800    
                                                                 
 dense_34 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 26s 4s/step - loss: 1.5799 - accuracy: 0.2068 - val_loss: 2.9212 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 18s 4s/step - loss: 1.5086 - accuracy: 0.3136 - val_loss: 3.2472 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 18s 4s/step - loss: 1.4580 - accuracy: 0.3614 - val_loss: 2.8698 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 16s 3s/step - loss: 1.4045 - accuracy: 0.4205 - val_loss: 2.5822 - val_accuracy: 0.0000e+00
Epoch 5/20
5/5 [==============================] - 18s 4s/step - loss: 1.3776 - accuracy: 0.4114 - val_loss: 2.6078 - val_accuracy: 0.0000e+00
Epoch 6/20
5/5 [==============================] - 16s 3s/step - loss: 1.3471 - accuracy: 0.4318 - val_loss: 2.9456 - val_accuracy: 0.0000e+00
Epoch 7/20
5/5 [==============================] - 12s 2s/step - loss: 1.3145 - accuracy: 0.4273 - val_loss: 3.0876 - val_accuracy: 0.0000e+00
18/18 [==============================] - 3s 144ms/step
3/3 [==============================] - 1s 184ms/step
18/18 [==============================] - 3s 162ms/step
3/3 [==============================] - 0s 121ms/step
Model: "sequential_38"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_35 (Embedding)    (None, 200, 50)           10000     
                                                                 
 spatial_dropout1d_35 (Spati  (None, 200, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_35 (LSTM)              (None, 200)               200800    
                                                                 
 dense_35 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 14s 2s/step - loss: 1.5693 - accuracy: 0.2341 - val_loss: 2.6625 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 10s 2s/step - loss: 1.5037 - accuracy: 0.3341 - val_loss: 3.2343 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 8s 2s/step - loss: 1.4508 - accuracy: 0.3864 - val_loss: 2.9858 - val_accuracy: 0.0091
Epoch 4/20
5/5 [==============================] - 10s 2s/step - loss: 1.4035 - accuracy: 0.4000 - val_loss: 2.6599 - val_accuracy: 0.0000e+00
Epoch 5/20
5/5 [==============================] - 10s 2s/step - loss: 1.3752 - accuracy: 0.4136 - val_loss: 2.6760 - val_accuracy: 0.0000e+00
Epoch 6/20
5/5 [==============================] - 8s 2s/step - loss: 1.3410 - accuracy: 0.4568 - val_loss: 3.1134 - val_accuracy: 0.0000e+00
Epoch 7/20
5/5 [==============================] - 10s 2s/step - loss: 1.3291 - accuracy: 0.4250 - val_loss: 3.0613 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 116ms/step
3/3 [==============================] - 0s 115ms/step
18/18 [==============================] - 4s 195ms/step
3/3 [==============================] - 0s 141ms/step
Model: "sequential_39"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_36 (Embedding)    (None, 200, 50)           10000     
                                                                 
 spatial_dropout1d_36 (Spati  (None, 200, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_36 (LSTM)              (None, 200)               200800    
                                                                 
 dense_36 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 14s 2s/step - loss: 1.5408 - accuracy: 0.2409 - val_loss: 3.0051 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 10s 2s/step - loss: 1.4878 - accuracy: 0.3409 - val_loss: 3.3543 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 9s 2s/step - loss: 1.4501 - accuracy: 0.3614 - val_loss: 2.7540 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 10s 2s/step - loss: 1.4031 - accuracy: 0.3932 - val_loss: 2.6403 - val_accuracy: 0.0000e+00
Epoch 5/20
5/5 [==============================] - 11s 2s/step - loss: 1.3824 - accuracy: 0.4045 - val_loss: 2.6085 - val_accuracy: 0.0000e+00
Epoch 6/20
5/5 [==============================] - 8s 2s/step - loss: 1.3439 - accuracy: 0.4545 - val_loss: 3.2857 - val_accuracy: 0.0000e+00
Epoch 7/20
5/5 [==============================] - 10s 2s/step - loss: 1.3295 - accuracy: 0.4250 - val_loss: 3.0760 - val_accuracy: 0.0273
Epoch 8/20
5/5 [==============================] - 10s 2s/step - loss: 1.2862 - accuracy: 0.4727 - val_loss: 3.5365 - val_accuracy: 0.0000e+00
18/18 [==============================] - 4s 117ms/step
3/3 [==============================] - 0s 112ms/step
18/18 [==============================] - 2s 117ms/step
3/3 [==============================] - 0s 107ms/step
Model: "sequential_40"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_37 (Embedding)    (None, 200, 50)           10000     
                                                                 
 spatial_dropout1d_37 (Spati  (None, 200, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_37 (LSTM)              (None, 200)               200800    
                                                                 
 dense_37 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 13s 2s/step - loss: 1.5489 - accuracy: 0.2364 - val_loss: 3.0371 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 10s 2s/step - loss: 1.4905 - accuracy: 0.3409 - val_loss: 3.2073 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 11s 2s/step - loss: 1.4508 - accuracy: 0.3568 - val_loss: 2.7607 - val_accuracy: 0.0000e+00
Epoch 4/20
5/5 [==============================] - 10s 2s/step - loss: 1.3995 - accuracy: 0.4250 - val_loss: 2.5513 - val_accuracy: 0.0000e+00
Epoch 5/20
5/5 [==============================] - 9s 2s/step - loss: 1.3720 - accuracy: 0.4295 - val_loss: 2.7156 - val_accuracy: 0.0000e+00
Epoch 6/20
5/5 [==============================] - 11s 2s/step - loss: 1.3390 - accuracy: 0.4523 - val_loss: 2.9742 - val_accuracy: 0.0182
Epoch 7/20
5/5 [==============================] - 12s 2s/step - loss: 1.3016 - accuracy: 0.4341 - val_loss: 3.2883 - val_accuracy: 0.0000e+00
18/18 [==============================] - 2s 115ms/step
3/3 [==============================] - 0s 121ms/step
18/18 [==============================] - 2s 115ms/step
3/3 [==============================] - 0s 116ms/step
Model: "sequential_41"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding_38 (Embedding)    (None, 200, 50)           10000     
                                                                 
 spatial_dropout1d_38 (Spati  (None, 200, 50)          0         
 alDropout1D)                                                    
                                                                 
 lstm_38 (LSTM)              (None, 200)               200800    
                                                                 
 dense_38 (Dense)            (None, 5)                 1005      
                                                                 
=================================================================
Total params: 211,805
Trainable params: 211,805
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/20
5/5 [==============================] - 13s 2s/step - loss: 1.5596 - accuracy: 0.2477 - val_loss: 2.9447 - val_accuracy: 0.0000e+00
Epoch 2/20
5/5 [==============================] - 10s 2s/step - loss: 1.5018 - accuracy: 0.3182 - val_loss: 3.3344 - val_accuracy: 0.0000e+00
Epoch 3/20
5/5 [==============================] - 10s 2s/step - loss: 1.4620 - accuracy: 0.3523 - val_loss: 3.0255 - val_accuracy: 0.0091
Epoch 4/20
5/5 [==============================] - 8s 2s/step - loss: 1.4243 - accuracy: 0.3795 - val_loss: 2.7202 - val_accuracy: 0.0000e+00
Epoch 5/20
5/5 [==============================] - 10s 2s/step - loss: 1.3831 - accuracy: 0.4295 - val_loss: 2.7481 - val_accuracy: 0.0000e+00
Epoch 6/20
5/5 [==============================] - 11s 2s/step - loss: 1.3455 - accuracy: 0.4477 - val_loss: 3.0522 - val_accuracy: 0.0000e+00
Epoch 7/20
5/5 [==============================] - 8s 2s/step - loss: 1.3268 - accuracy: 0.4386 - val_loss: 3.0525 - val_accuracy: 0.0091
18/18 [==============================] - 4s 197ms/step
3/3 [==============================] - 0s 112ms/step
18/18 [==============================] - 2s 116ms/step
3/3 [==============================] - 0s 117ms/step
Result of all runs:   model  train accuracy  test accuracy  train F1 score  test F1 score
0  LSTM        0.392727       0.289157        0.341784       0.271048
1  LSTM        0.407273       0.325301        0.337902       0.315034
2  LSTM        0.390909       0.313253        0.321514       0.300340
3  LSTM        0.389091       0.301205        0.331512       0.279528
4  LSTM        0.389091       0.313253        0.334186       0.301020

Conclusion-

1. Glove dataset performed best in LSTM model. F1 score is reaching 30%. Accuracy is around 32% for majority of runs.

2. Saving the model for future purpose. It is saved to "/content/drive/MyDrive/Colab Notebooks/Projects/NLP/glove_model.h5"

3. ANN model has performed really well compared to LSTM. ANN had reached 45% accuracy and 40% F1 score.

End of Execution. Thank you.